-- Logs begin at Mon 2022-08-01 13:17:59 UTC, end at Mon 2022-08-01 14:39:58 UTC. -- Aug 01 14:04:29.305998 dv-cinder-845990-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 14:04:29.316560 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 14:04:29 2022] *** Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: nodename: dv-cinder-845990-5-iscsi Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: machine: x86_64 Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: clock source: unix Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: pcre jit disabled Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: detected number of CPU cores: 4 Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: current working directory: / Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: detected binary path: /usr/bin/uwsgi-core Aug 01 14:04:29.319084 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: your processes number limit is 47680 Aug 01 14:04:29.319801 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: your memory page size is 4096 bytes Aug 01 14:04:29.319801 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: detected max file descriptor number: 2048 Aug 01 14:04:29.319801 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: lock engine: pthread robust mutexes Aug 01 14:04:29.319801 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: thunder lock: enabled Aug 01 14:04:29.319801 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 14:04:29.319801 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 14:04:29.359483 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: Python main interpreter initialized at 0x5652507fc9c0 Aug 01 14:04:29.359483 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: python threads support enabled Aug 01 14:04:29.359483 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: your server socket listen backlog is limited to 100 connections Aug 01 14:04:29.359483 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: your mercy for graceful operations on workers is 90 seconds Aug 01 14:04:29.360223 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 14:04:29.360223 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: *** Operational MODE: preforking *** Aug 01 14:04:29.360223 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: *** uWSGI is running in multiple interpreter mode *** Aug 01 14:04:29.360223 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: spawned uWSGI master process (pid: 80517) Aug 01 14:04:29.360443 dv-cinder-845990-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 14:04:29.361742 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: spawned uWSGI worker 1 (pid: 80524, cores: 1) Aug 01 14:04:29.362968 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: spawned uWSGI worker 2 (pid: 80525, cores: 1) Aug 01 14:04:29.362968 dv-cinder-845990-5-iscsi devstack@keystone.service[80517]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 14:04:35.103191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:04:35.103191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: from cryptography.utils import int_from_bytes Aug 01 14:04:35.104143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:04:35.104143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: from cryptography.utils import int_from_bytes Aug 01 14:04:35.104568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:04:35.104568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: from cryptography.utils import int_from_bytes Aug 01 14:04:35.104676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:04:35.104676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: from cryptography.utils import int_from_bytes Aug 01 14:04:35.155731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 14:04:35.155923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 14:04:35.155999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 14:04:35.156096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] command line args: [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 14:04:35.156165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 14:04:35.156310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] command line args: [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 14:04:35.156394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 14:04:35.156529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ================================================================================ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 14:04:35.156623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 14:04:35.156728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ================================================================================ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 14:04:35.156803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] admin_token = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.156994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] admin_token = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config_dir = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config_dir = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config_source = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] debug = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config_source = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] debug = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.157876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.158870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_config_append = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_config_append = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_dir = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_dir = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.159856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.160997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.161976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.162924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.163106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.163215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.163643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.163731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.163864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] publish_errors = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.163941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] publish_errors = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.164956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] standard_threads = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] standard_threads = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.165977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_journal = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_json = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_stderr = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_journal = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_syslog = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_json = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_stderr = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.166965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_syslog = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.167189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:04:35.167289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.167442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.167553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.167705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.167806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.167969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.168963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.169911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.170938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.171051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.171170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.171385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.171648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.171851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.171989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.172119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.172250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.172391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.172521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.173979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.174941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.175945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.176921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.177896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.178954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.179895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.180912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.181969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.182905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.183918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.184878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.185987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.186935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.187969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.188125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.188210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.188961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.189218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.189482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.189825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.190176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.190464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.190703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.190940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.191928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.192945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.external = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.193956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.password = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.token = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.194965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.195948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.external = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.196913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.password = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.token = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.197898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.198916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.199923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.200990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.201961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.202974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.203992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.204926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.205885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.206840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.207849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.208962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.209925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.210893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.211876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.212943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.213928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.214976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.215947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.216891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.217984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.218950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.219906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.220869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.221946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.222965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.223868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.224049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.224121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.224289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.224359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.224783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.224859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.225981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.226986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.227953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.228135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.228216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.228384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.228607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.228806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.228910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.229042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.229125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.229404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.229662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.229802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.229928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.230905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.231129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.231345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.231436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.231596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.231659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.231865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.232907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.233843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.234963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.235051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.235757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.235848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.236937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.237988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.238918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.239914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.240987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.241964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.242915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.243956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.244900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.245952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 14:04:35.246051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.246243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.246474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.246705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.246939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.247180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.247414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.247679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.247912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.248158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.248396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.248630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.248857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.249093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.249326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:04:35.249495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 14:04:35.276311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.276540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.276747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.279451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.279709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.279938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.285423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.288786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.317522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.317752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.319380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.319631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.374314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.374838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.427625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.427829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.427933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.428016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.496362 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.496589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.496811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.497841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.498006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.498226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.522976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.523109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.523230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.523348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.523422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.523562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.523778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.535590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.535809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.549534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.549775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.549976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.550137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.550280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.550422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.550561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.550701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.550836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.551034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.551171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.551302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.551302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.551586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:04:35.568444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.568688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.568999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:04:35.569123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:04:35.569123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.569386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.569512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:04:35.569637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:04:35.569759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.569890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.570026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.570145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.570268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.570390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.570511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.570644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.580055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.580343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.580617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.580905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.581128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.581422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.581693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.581971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.587896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.588899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.592757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.593047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.593327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.593610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.593894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.594055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.594190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.594317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.594444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.595856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.596000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.596115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.596298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.596569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.596839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.597099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.622740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.623231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.623812 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.624182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.624855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.625213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.629121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.629401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.629689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.629975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.630257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.630543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.636808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.637169 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.638285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.638673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.643421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.643756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.644017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.644255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.644514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.644744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.644744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.645172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.645393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.645641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.645882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.646080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.646278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.646474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.646474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.646474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.662059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.662373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.662604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.663005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.666573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.667005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.669952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.670208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.670403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.670557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.675515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.675983 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.678702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.678877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.681863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.682998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.683309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.683449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.683560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.699405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.699725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.699994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.700183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.700328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.700485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.700748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.701322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.707185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.708324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.708776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.709050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.709863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.710133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.711892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.712175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.712477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.712759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.713056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.713207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.713391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.713751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.714108 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.714489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.725379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.725670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.725955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.726235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.726518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.726792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.727087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.727184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.727404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.727708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.727984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.728259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.728528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.728799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.743346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.745280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.746369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.748300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.749344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.751270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.752144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.753696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.754013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.754302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.754438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.756259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.756670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.756978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.761007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.762643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.763434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.765068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.765582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.765874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.766164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.766986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.768567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.768857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.769145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.769431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.775628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.775960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.776228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.778495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.778802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.779080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.783958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.784261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.784549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.784833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.785116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.785399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.785682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.785953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.786352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.786643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.786778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:04:35.786925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.787838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.788028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.788316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.788580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.788957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.789242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.789529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.790080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.790288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.790564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:04:35.821478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5652507fc9c0 pid: 80525 (default app) Aug 01 14:04:35.824350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5652507fc9c0 pid: 80524 (default app) Aug 01 14:04:35.825062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b08bb-2845-44e3-86cf-51c4e1170f86 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:35.825354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b08bb-2845-44e3-86cf-51c4e1170f86 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:35.825633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b08bb-2845-44e3-86cf-51c4e1170f86 None None] PATH_INFO: `/v3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:35.830573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Mon Aug 1 14:04:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:43.815403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:43.816002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:43.816002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:43.820446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:43.826769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:43.827293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:43.827293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:43.898679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: INFO dbcounter [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] Registered counter for database keystone Aug 01 14:04:43.919672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG oslo_db.sqlalchemy.engines [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80525) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 14:04:43.927310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writer thread running {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 14:04:44.123150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG passlib.handlers.bcrypt [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80525) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 14:04:44.123794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG passlib.handlers.bcrypt [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80525) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 14:04:44.181208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:44.182781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:44.339197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 01 14:04:44.480031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:44.551132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: INFO dbcounter [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] Registered counter for database keystone Aug 01 14:04:44.573950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG oslo_db.sqlalchemy.engines [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80524) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 14:04:44.581765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writer thread running {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 14:04:44.713344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fa4c8-6c75-4e7d-8e79-4163a3acdb5e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:44.718215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:44.718947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:44.719305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:44.719661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:44.729947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:44.784869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 14:04:44.785227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9236e-12a3-4ddf-8ed3-f78e978b2a44 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:44.796981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Aug 1 14:04:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:04:44.806248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:44.823035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2d7fd-0f38-4461-95a6-9903675ff08c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:44.827649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:44.828916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:44.829200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:44.829473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] PATH_INFO: `/v3/projects/045f384688624ccf8339b3c71e2bd890` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:44.831107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] RBAC: Authorizing `identity:get_project(project_id=045f384688624ccf8339b3c71e2bd890)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:44.883623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 14:04:44.883960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99d93e-edc0-4509-971f-1aa7d713a6fa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:44.888972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Aug 1 14:04:44 2022] GET /identity/v3/projects/045f384688624ccf8339b3c71e2bd890 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:47.183054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061afb04-99fe-488c-8c76-117d2626c735 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:47.183650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061afb04-99fe-488c-8c76-117d2626c735 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:47.183650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061afb04-99fe-488c-8c76-117d2626c735 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:47.187805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:47.196136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:47.197131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:47.197131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:47.230130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:47.232005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:47.241676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 14:04:47.266224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061afb04-99fe-488c-8c76-117d2626c735 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:47.279757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061afb04-99fe-488c-8c76-117d2626c735 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:47.284554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52360289-ee3d-4649-adc5-6f93917f7a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:47.285208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52360289-ee3d-4649-adc5-6f93917f7a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:47.285496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52360289-ee3d-4649-adc5-6f93917f7a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:47.285797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52360289-ee3d-4649-adc5-6f93917f7a98 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:47.295611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52360289-ee3d-4649-adc5-6f93917f7a98 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:47.297980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52360289-ee3d-4649-adc5-6f93917f7a98 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:47.308479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Aug 1 14:04:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:04:47.317236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:47.336037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bff7b6-6d99-4c24-ba28-e921a7e0fdd4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:47.343702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b10cf-d905-49b5-85c0-13ac98b9f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:47.344912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b10cf-d905-49b5-85c0-13ac98b9f4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:47.345199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b10cf-d905-49b5-85c0-13ac98b9f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:47.345474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b10cf-d905-49b5-85c0-13ac98b9f4a8 admin admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:47.346992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b10cf-d905-49b5-85c0-13ac98b9f4a8 admin admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:47.349045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b10cf-d905-49b5-85c0-13ac98b9f4a8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:47.351107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Mon Aug 1 14:04:47 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:04:49.594800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbdb1c-19a2-44e5-8ae3-2fb9891c4554 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:49.595636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbdb1c-19a2-44e5-8ae3-2fb9891c4554 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:49.595636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbdb1c-19a2-44e5-8ae3-2fb9891c4554 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:49.596763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:49.603341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:49.603969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:49.603969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:49.629387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:49.630683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:49.745254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 6/11] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:04:49.764340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3f650-e517-44bf-a7dc-ae6eae86507e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:49.764631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3f650-e517-44bf-a7dc-ae6eae86507e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:49.764918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3f650-e517-44bf-a7dc-ae6eae86507e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:49.766842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 6/12] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:49.770359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:49.772349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:49.772636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:49.772908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:49.803872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:49.826450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG passlib.handlers.bcrypt [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80524) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 14:04:49.826903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG passlib.handlers.bcrypt [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80524) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 14:04:49.876044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:49.878023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:49.908163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa4711-4c10-4e32-8568-5bd8441819d7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:49.961604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:49.962266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:49.962564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:49.962838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:49.976093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:04:49.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:04:49.988507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:04:49.989965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 7/13] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:04:49.994079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.011677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 7/14] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:04:50.035887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.108228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:50.142752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964256d-dc43-4fa7-93b5-f3bbc512eb49 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.182839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea6407-f02e-4313-a240-131082ac0247 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.187730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:50.200945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.202645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.203173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.203722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.204733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:50.205297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:50.207005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:50.220927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:04:50.229744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.251001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.251949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.252238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.252513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.368611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:50.370466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:50.509242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f251b5-5feb-47c7-91a2-be06545d79a1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.548360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:04:50.549718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.554989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:50.569436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.570123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.570416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.570764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.573050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:50.575886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:50.583506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:04:50 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:50.587641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.607625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:50.692024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0b8e-e6f5-4af5-b72e-6430612fd8fa None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.729486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86aed-3394-4c6c-9383-a2c66379ffe6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.743428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.745159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.745475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.745874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.767340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:50.768822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:50.772226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.772887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.773171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.773453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.775954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:50.777916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:50.779695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:04:50.786740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:04:50.795886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:04:50.797589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:04:50.798996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:04:50.798996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:04:50.798996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:04:50.798996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:04:50.798996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:04:50.808134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:50.878915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:50.909649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:50.941494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017171-7807-4d91-9717-80c7f14589a5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.945678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4406d-16d5-4695-9e12-b521ad679d4b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:50.986473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.987119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.987406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.987717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.988506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:50.988813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:50.990286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444d644-3385-47b3-b3ce-30fc8adfe83f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:50.996419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:50.997485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:50.997928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:50.998341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:50.999171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.domain.id=default, user.id=65c8d33b0a3b4ecb82a05c93cec78423) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:50.999513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:51.001660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b33d6-f1bb-4940-9ac7-5c953587945f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:51.009741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Aug 1 14:04:50 2022] GET /identity/v3/role_assignments?user.id=65c8d33b0a3b4ecb82a05c93cec78423&scope.domain.id=default&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 233 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:04:51.016322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:04:53.860307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e956-6826-4a24-b508-6a69a20322c3 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:53.861578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e956-6826-4a24-b508-6a69a20322c3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:53.862094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e956-6826-4a24-b508-6a69a20322c3 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:53.863739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:53.869720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7ded7-d548-476f-a89f-17fb4d8508a6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:53.870698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7ded7-d548-476f-a89f-17fb4d8508a6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:53.871298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7ded7-d548-476f-a89f-17fb4d8508a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:53.900306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-13c7ded7-d548-476f-a89f-17fb4d8508a6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:53.902721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-13c7ded7-d548-476f-a89f-17fb4d8508a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:54.013421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 11/23] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:04:54.049052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619e956-6826-4a24-b508-6a69a20322c3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:54.196432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85133a32-11e3-43db-8ddc-8214c1dcd4a9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.197143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85133a32-11e3-43db-8ddc-8214c1dcd4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.197634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85133a32-11e3-43db-8ddc-8214c1dcd4a9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.199376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 12/24] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:04:54.209670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619e956-6826-4a24-b508-6a69a20322c3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:54.214139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.214876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.215359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.256053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:54.257040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:54.257746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.258114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.258472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.258744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:54.275380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:04:54.283717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:04:54.284614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:04:54.285490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:04:54.288214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 13/25] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:04:54.297139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:54.303207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:54.408112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:54.467567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:54.513218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:54.646601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e58a7-d699-4c3f-a5b9-0ab159e73131 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:54.691413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1b5d5-7073-44ea-a544-bbe353c8cf6c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:54.734290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:54.736299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.737005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.737438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.738892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:54.739827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:54.742051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a87b9a-edf8-422d-bd80-a13f058303bb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:54.753359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd4551-5aee-4d27-9274-91b61a7a9e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:54.753953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 14/27] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:54.754081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4551-5aee-4d27-9274-91b61a7a9e98 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.754332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4551-5aee-4d27-9274-91b61a7a9e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.754621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4551-5aee-4d27-9274-91b61a7a9e98 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.755422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd4551-5aee-4d27-9274-91b61a7a9e98 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:54.756208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf24f26-7642-4b6a-b114-2bf39d706d43 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.756729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf24f26-7642-4b6a-b114-2bf39d706d43 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.756829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd4551-5aee-4d27-9274-91b61a7a9e98 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:54.757288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf24f26-7642-4b6a-b114-2bf39d706d43 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.759180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 15/28] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:54.760823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c2efa-2f68-446c-be40-0f67f38f4585 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.762012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c2efa-2f68-446c-be40-0f67f38f4585 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.762694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c2efa-2f68-446c-be40-0f67f38f4585 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.766227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 16/29] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:54.767436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c2efa-2f68-446c-be40-0f67f38f4585 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:54.888259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:54.936928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c2efa-2f68-446c-be40-0f67f38f4585 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:54.991617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf031e-1c9b-4968-9722-151df6b52e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:54.992623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf031e-1c9b-4968-9722-151df6b52e39 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:54.993204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf031e-1c9b-4968-9722-151df6b52e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:54.993689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf031e-1c9b-4968-9722-151df6b52e39 None admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:54.995307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf031e-1c9b-4968-9722-151df6b52e39 None admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:54.997675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf031e-1c9b-4968-9722-151df6b52e39 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:55.001219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 17/30] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:04:54 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:55.003167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.003700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.004182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.027648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:55.029325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:55.096488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 14/31] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Aug 1 14:04:54 2022] POST /identity/v3/roles => generated 219 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:04:55.101132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.101491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.101843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.126719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:55.131545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:55.153204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 18/32] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:55.155230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:55.262286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:55.281002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:55.340477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabe953-80da-4da2-a9b4-2b95454a29d4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:55.390243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:55.392092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.392909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.393543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.395753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:55.398258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:55.400998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 19/33] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:55.403003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:55.455656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 15/34] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:55.458559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:55.584294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4cbf-947d-49ea-aa2c-7892ad0f861d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:55.647782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:55.664702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:55.667015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.668315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.669461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.672776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a239-507b-4cca-9f5b-815994e96f41 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:55.694252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:55.698961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:55.728461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6c78b-7778-4cae-82e3-f65c517b6d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:55.729159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6c78b-7778-4cae-82e3-f65c517b6d48 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.729446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6c78b-7778-4cae-82e3-f65c517b6d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.729862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6c78b-7778-4cae-82e3-f65c517b6d48 None admin] PATH_INFO: `/v3/domains/default/users/65c8d33b0a3b4ecb82a05c93cec78423/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:04:55.730000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:04:55.731447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:04:55.733793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:04:55.733793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:04:55.733793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:04:55.733793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:04:55.733793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 20/35] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:04:55.733793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:55.735591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6c78b-7778-4cae-82e3-f65c517b6d48 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=65c8d33b0a3b4ecb82a05c93cec78423, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:55.738463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6c78b-7778-4cae-82e3-f65c517b6d48 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:55.764432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 16/36] 10.222.0.17 () {58 vars in 1361 bytes} [Mon Aug 1 14:04:55 2022] PUT /identity/v3/domains/default/users/65c8d33b0a3b4ecb82a05c93cec78423/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:04:55.766793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c281e01-e1f2-4549-8d0d-1b51365576dc None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.767199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c281e01-e1f2-4549-8d0d-1b51365576dc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.767753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c281e01-e1f2-4549-8d0d-1b51365576dc None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.769806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 17/37] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:55.770672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c281e01-e1f2-4549-8d0d-1b51365576dc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:55.826362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:55.917946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb341d-5c2f-4786-b506-81b4e3caa773 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:55.945867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c281e01-e1f2-4549-8d0d-1b51365576dc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:55.976255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd4bb1-6968-4bda-9253-55678760be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:55.977321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd4bb1-6968-4bda-9253-55678760be24 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:55.977805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd4bb1-6968-4bda-9253-55678760be24 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:55.978301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd4bb1-6968-4bda-9253-55678760be24 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:55.979558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd4bb1-6968-4bda-9253-55678760be24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:55.983039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd4bb1-6968-4bda-9253-55678760be24 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:56.007046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:56.007993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.008274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.008550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.009266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:56.009571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:56.010978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7329fb0-4d9c-4344-8386-8b02accd4192 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:56.028611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 18/38] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:56.031916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5336fcf-8554-47eb-a4a1-3570843eb82f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.031916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5336fcf-8554-47eb-a4a1-3570843eb82f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.031916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5336fcf-8554-47eb-a4a1-3570843eb82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.050837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a5336fcf-8554-47eb-a4a1-3570843eb82f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:56.052266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a5336fcf-8554-47eb-a4a1-3570843eb82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:56.143818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 19/39] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:56.146117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f54f-d7e7-4347-a9b7-29fed12039fb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.146716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f54f-d7e7-4347-a9b7-29fed12039fb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.147277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f54f-d7e7-4347-a9b7-29fed12039fb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.149147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 20/40] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:56.150242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f54f-d7e7-4347-a9b7-29fed12039fb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:56.185158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:56.218706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 21/41] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Aug 1 14:04:55 2022] POST /identity/v3/roles => generated 225 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:04:56.222109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.223559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.223559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.257867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:56.260656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:56.340197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f54f-d7e7-4347-a9b7-29fed12039fb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:56.351548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:56.401262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:56.402529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.403130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.403814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:56.404159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.404849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:56.405528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:56.407370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:56.461698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 21/43] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:04:56 2022] POST /identity/v3/projects => generated 301 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:04:56.463083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:56.595720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:56.614461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5aa3b-14f3-457d-94e1-db3383d48a84 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:56.666277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf11cc-628a-4ef4-ad50-a82ec6b9be90 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:56.694720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:56.694720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.694720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.694720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.695982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:56.697935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:56.700805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 23/44] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:56.708869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:56.725554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5f86c-e427-4af9-ab3b-361933597322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:56.726102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5f86c-e427-4af9-ab3b-361933597322 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.726422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5f86c-e427-4af9-ab3b-361933597322 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.726804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5f86c-e427-4af9-ab3b-361933597322 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.727603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5f86c-e427-4af9-ab3b-361933597322 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:56.729018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5f86c-e427-4af9-ab3b-361933597322 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:56.755364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 22/45] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Aug 1 14:04:56 2022] POST /identity/v3/roles => generated 223 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:04:56.798311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:56.896747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fa0d0-7725-4562-b4f4-8ac2ff704ea2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:56.942930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59f1ac-5d11-478f-8562-7beacbddd8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:56.944071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59f1ac-5d11-478f-8562-7beacbddd8e0 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:56.944504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59f1ac-5d11-478f-8562-7beacbddd8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:56.944800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59f1ac-5d11-478f-8562-7beacbddd8e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:56.945582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59f1ac-5d11-478f-8562-7beacbddd8e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:56.947167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59f1ac-5d11-478f-8562-7beacbddd8e0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:56.999573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:04:56 2022] POST /identity/v3/projects => generated 312 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:04:58.523579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7159a-339b-4e06-88ef-49c86e0ba499 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:58.524214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7159a-339b-4e06-88ef-49c86e0ba499 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:58.524471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7159a-339b-4e06-88ef-49c86e0ba499 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:58.526767 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:58.533813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:58.534329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:58.534409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:58.554990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:58.556417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:58.634983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:58.668242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7159a-339b-4e06-88ef-49c86e0ba499 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:58.818350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7159a-339b-4e06-88ef-49c86e0ba499 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:58.862261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:58.873285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:58.873963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:58.874251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:58.874537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:58.897680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:04:58.910204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:04:58.911119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:04:58.911912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:04:58.912476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 24/49] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:04:58.922944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:58.977821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:59.111691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c60752-08dd-465b-8eff-85c747eb5972 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:59.175865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:59.176512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.176797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.177076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.177820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:59.178126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:59.179767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f6a8f-c207-447b-88ab-2ee879f30e1d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:59.195814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 26/50] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:59.203907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:59.222755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8b5b9-4316-4a57-aedc-8d8a736e91f9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.223093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8b5b9-4316-4a57-aedc-8d8a736e91f9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.223378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8b5b9-4316-4a57-aedc-8d8a736e91f9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.225957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 27/51] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:04:59.234690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.235204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.235761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.266589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:04:59.268792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:04:59.358134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca7083-ea17-461c-935d-1a0f176da817 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:59.392139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 28/52] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:04:59.403353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:59.404296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.404453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.404738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.406390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:59.408693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:59.410790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 25/53] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:04:59 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:59.416048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:59.416478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:59.437038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:59.492501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:59.556321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541606f-1615-429f-a12f-ab6f63f1d610 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:59.568674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb5435-09c3-41fc-9ef4-a76dff29b9dc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:59.601771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:59.602722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.603107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.603610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.605151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:59.606896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:59.608676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:04:59.611708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:59.613613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.614140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.614641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.616824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:59.618751 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:59.620487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 29/55] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:04:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:59.621239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:59.627038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:04:59.751979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:59.772928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629b26-d4c6-4ba5-bbba-2eecbed17c56 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:59.773439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b9b74-f18e-41bd-b24f-f8c079467561 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:04:59.803324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:04:59.816434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:59.817074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.817357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.817635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.818020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88740ad-20cc-4c71-a9f9-2fe4eacdc132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:04:59.818446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.domain.id=default, user.id=65c8d33b0a3b4ecb82a05c93cec78423) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:04:59.818734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88740ad-20cc-4c71-a9f9-2fe4eacdc132 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:04:59.818881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:59.818975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88740ad-20cc-4c71-a9f9-2fe4eacdc132 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:04:59.819322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88740ad-20cc-4c71-a9f9-2fe4eacdc132 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:04:59.820079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88740ad-20cc-4c71-a9f9-2fe4eacdc132 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:04:59.820366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e099e8-5a1a-4d39-bf24-a54ff4d192ff None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:59.821571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88740ad-20cc-4c71-a9f9-2fe4eacdc132 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:04:59.832588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 27/56] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Aug 1 14:04:59 2022] GET /identity/v3/role_assignments?user.id=65c8d33b0a3b4ecb82a05c93cec78423&scope.domain.id=default&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 530 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:04:59.862427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 30/57] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Aug 1 14:04:59 2022] POST /identity/v3/projects => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:05:02.006344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93c316-0ad0-4c45-a77b-07ef3052f790 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.006887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93c316-0ad0-4c45-a77b-07ef3052f790 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.006887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93c316-0ad0-4c45-a77b-07ef3052f790 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.008323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:02.014217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.014743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.014954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.035396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:02.036666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:02.112828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:02.136942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93c316-0ad0-4c45-a77b-07ef3052f790 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:02.264138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93c316-0ad0-4c45-a77b-07ef3052f790 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:02.300891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15183696-d4a6-4e34-84c3-0f11d1b08821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:02.301540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15183696-d4a6-4e34-84c3-0f11d1b08821 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.301826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15183696-d4a6-4e34-84c3-0f11d1b08821 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.302100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15183696-d4a6-4e34-84c3-0f11d1b08821 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.303275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15183696-d4a6-4e34-84c3-0f11d1b08821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:02.304773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15183696-d4a6-4e34-84c3-0f11d1b08821 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:02.306395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:02.312184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:02.382254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:02.420086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bdf4a-3024-4682-a311-bca1054ad03b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:02.460028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa3b9e-812d-4826-9ff9-d87a813a7d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:02.460771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa3b9e-812d-4826-9ff9-d87a813a7d96 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.461139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa3b9e-812d-4826-9ff9-d87a813a7d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.461514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa3b9e-812d-4826-9ff9-d87a813a7d96 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.462298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa3b9e-812d-4826-9ff9-d87a813a7d96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:02.463947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa3b9e-812d-4826-9ff9-d87a813a7d96 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:02.495336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:02 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:06.412844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48094993-fa12-41f7-980f-e5f469eb4a92 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.414041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48094993-fa12-41f7-980f-e5f469eb4a92 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.414564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48094993-fa12-41f7-980f-e5f469eb4a92 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.416493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:06.424717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8322241-f385-4fd7-bc71-3b3ae46cec96 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.425735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8322241-f385-4fd7-bc71-3b3ae46cec96 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.426216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8322241-f385-4fd7-bc71-3b3ae46cec96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.452596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f8322241-f385-4fd7-bc71-3b3ae46cec96 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:06.454841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f8322241-f385-4fd7-bc71-3b3ae46cec96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:06.565158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 33/63] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:06.606590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054ef20-bb86-4bbf-883c-da3cc8a9c100 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.607210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054ef20-bb86-4bbf-883c-da3cc8a9c100 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.607715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054ef20-bb86-4bbf-883c-da3cc8a9c100 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.609842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 31/64] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:06.615493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234f3a-a333-4a7c-8cff-f73d19d28b06 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.615807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234f3a-a333-4a7c-8cff-f73d19d28b06 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.616318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234f3a-a333-4a7c-8cff-f73d19d28b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.630081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054ef20-bb86-4bbf-883c-da3cc8a9c100 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:06.652635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-18234f3a-a333-4a7c-8cff-f73d19d28b06 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:06.653908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-18234f3a-a333-4a7c-8cff-f73d19d28b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:06.736724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:06.804720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:06.863296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 34/65] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:06.864867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad443-d3de-4a1a-9c67-e56fd922ad6f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.865395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad443-d3de-4a1a-9c67-e56fd922ad6f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.865817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad443-d3de-4a1a-9c67-e56fd922ad6f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.867267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 35/66] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:06.872817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.873295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.873744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.888624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054ef20-bb86-4bbf-883c-da3cc8a9c100 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:06.899954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:06.901502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:06.962947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:06.964080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:06.964587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:06.965083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:06.987215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:07.003647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:07.005362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:07.006932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:07.008542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 32/67] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:07.010395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.042062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 36/68] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:07.044674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.226983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3d98a-a990-422e-941f-e1466147e835 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.243874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:07.244116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252c19a-c40c-4b6f-aa78-2673571bffa0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.267583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:07.270653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.271324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.271653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.271933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.272677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:07.272984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:07.274412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:07.283354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 37/69] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:07.284223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.291577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa2903-7c7b-4fb8-9530-d7ed4c83f448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.292240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2903-7c7b-4fb8-9530-d7ed4c83f448 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.292526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2903-7c7b-4fb8-9530-d7ed4c83f448 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.292807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2903-7c7b-4fb8-9530-d7ed4c83f448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.305525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa2903-7c7b-4fb8-9530-d7ed4c83f448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f0aa2903-7c7b-4fb8-9530-d7ed4c83f448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:07.310934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:07.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:07.313469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 33/70] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:07.314141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e0fe1-4f81-4be3-b500-746aeda67b5f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.314448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e0fe1-4f81-4be3-b500-746aeda67b5f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.314681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e0fe1-4f81-4be3-b500-746aeda67b5f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.316172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 34/71] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:07.317459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a133e-81b6-4d7b-915f-539cda273632 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.317904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a133e-81b6-4d7b-915f-539cda273632 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.318184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a133e-81b6-4d7b-915f-539cda273632 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.319548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 35/72] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:07.324027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a133e-81b6-4d7b-915f-539cda273632 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.442016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c763a-ed72-4809-8246-97a59aa7e888 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.451162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a133e-81b6-4d7b-915f-539cda273632 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.484481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.485138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.485418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.485686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.495321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975ad3-40af-4743-bbf6-2599065703e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.496013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975ad3-40af-4743-bbf6-2599065703e9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.496381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975ad3-40af-4743-bbf6-2599065703e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.496738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975ad3-40af-4743-bbf6-2599065703e9 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.498670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:07.508843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:07.510616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:07.512192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.525365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975ad3-40af-4743-bbf6-2599065703e9 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:07.526883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975ad3-40af-4743-bbf6-2599065703e9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:07.528663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 36/74] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:07.531497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ae791-0312-4049-891d-a594b390d571 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.531946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ae791-0312-4049-891d-a594b390d571 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.532386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ae791-0312-4049-891d-a594b390d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.552234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:07.553478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a08ae791-0312-4049-891d-a594b390d571 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:07.555236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a08ae791-0312-4049-891d-a594b390d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:07.587196 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:07.645489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 37/75] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:07.648859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.649154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.649439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.665023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d23d-e564-4958-b9d5-277561d14728 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.674507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:07.675769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:07.720135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.721574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.722512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.722512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.723023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:07.724719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:07.725131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:07.735490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:07.736368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.769467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 38/77] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:07.769467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:07.891781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:07.896958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:07.899200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495c09c-0a1c-4030-87b4-192cdcb278ff None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.908496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240c561-9bd3-40f1-b992-56235e9b373a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:07.945049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.946307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.946491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.947216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.948415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:07.948897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:07.951240 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:07.954568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:07.955743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:07.956053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:07.956314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:07.963044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:07.964093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.043838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.046746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.120121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5362cb-ec66-46b6-b46b-aaf09a9365b3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.160643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.161445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.161801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.162167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.163710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.165281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.167170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 41/79] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:08.168542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.178410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:08.189200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 39/80] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:08.190058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.217327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:08.329239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b6ec-b53d-4270-bf9c-6d377b9853fe None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.342467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9c0d-2093-4198-ab91-dfb9e610ccff None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.369006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381165-bfab-4773-bc83-969340a174ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.369671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381165-bfab-4773-bc83-969340a174ce None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.369957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381165-bfab-4773-bc83-969340a174ce None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.370233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381165-bfab-4773-bc83-969340a174ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.382791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381165-bfab-4773-bc83-969340a174ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:08.382996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.383534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.383823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.384103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.385601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.387141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-57381165-bfab-4773-bc83-969340a174ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:08.388754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:08.398093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:08.398997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:08.400013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 42/81] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:08 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:08.400128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 40/82] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:08.400239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.400382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381165-bfab-4773-bc83-969340a174ce None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.473464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:08.524205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:08.534260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661894c6-b107-41be-9f70-65bd4b273019 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.541569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381165-bfab-4773-bc83-969340a174ce None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.575881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.576568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.576879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.577162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.579058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.580775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.582437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 43/83] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:08 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:08.589958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.592277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.592856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.593234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.593533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.594470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.596123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.626247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 41/84] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:08 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:08.627129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.739479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708390d-21a4-4404-b6d9-2cbf2966e1f1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.776680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112888c-e21d-48da-97c5-7ca0b5642276 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:08.793514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:08.801642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.801642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.802018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.802018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.804230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.805798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.808931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 44/85] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:08 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:08.808931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:08.837095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:08.837670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:08.837974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:08.838250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:08.863280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:08.951500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:08.953277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:08.994963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b19cc-d727-4f76-9320-15e4bc768219 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:09.045402 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:09.046062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:09.046525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:09.046822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:09.047634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:09.048133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:09.049733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:09.060294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 45/86] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:09.061270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:09.130158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 42/87] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:09.131569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:09.152318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:09.180750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:09.192576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a0e26-2055-4ce6-b64c-fbb28702be15 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:09.232629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:09.233487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:09.233883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:09.234234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:09.310139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ada477-de39-4f11-a2ab-48529ebebd14 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:09.341269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:09.342787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:09.366558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:09.367323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:09.367645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:09.367923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:09.369147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=b0b3f354b8b6415a9dda8dd637e5335e) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:09.370038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:09.371764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:09.380448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 43/88] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:09 2022] GET /identity/v3/role_assignments?user.id=b0b3f354b8b6415a9dda8dd637e5335e&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=d47409cb4aa642018317ecedd3a36461 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:09.381693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:09.485167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:09.527720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe994f-d1e8-48ee-b343-41e3fca49fd9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:09.574748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:09.575443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:09.575784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:09.576064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:09.578099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:09.579662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:09.583815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 44/89] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:09 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:09.585608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:09.646590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:09.667161 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 46/90] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:09.668747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:09.828661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad5a0-1227-4984-824b-de7e55e96882 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:09.846400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a74d30-a869-49d5-b2fd-ba68104aa5b8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:09.884355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:09.885745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:09.886657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:09.887331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:09.951006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:09.951657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:09.951965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:09.952623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:09.955363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:09.967493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:09.970368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 45/91] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:09 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:09.971439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:09.971974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:09.986693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:09.988241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:10.000609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:10.141193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 47/92] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:10.142761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:10.275194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:10.286955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9e04-5290-49c8-9431-186082d11c41 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:10.289746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618e42b-f87c-4ab7-8fb7-c68fac5d7fc2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:10.337510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:10.338520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:10.338817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:10.339098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:10.340014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=65c8d33b0a3b4ecb82a05c93cec78423) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:10.343592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:10.346249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:10.361220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 48/93] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:10 2022] GET /identity/v3/role_assignments?user.id=65c8d33b0a3b4ecb82a05c93cec78423&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:10.362732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:10.377545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:10.378268 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:10.378561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:10.378845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:10.380346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:10.381817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:10.383604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 46/94] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:09 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:10.392073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:10.617458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:10.677625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4667-0f1d-427f-b681-e24050d75624 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:10.697752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510df8c-2961-479a-b0f9-df90851ca3fc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:10.767030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:10.768682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:10.769457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:10.770194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:10.772032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] RBAC: Adding query filter params (role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=b0b3f354b8b6415a9dda8dd637e5335e) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:10.772210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:10.772878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:10.773168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:10.773445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] PATH_INFO: `/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:10.773978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:10.774910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] RBAC: Authorizing `identity:get_project(project_id=96f902ee515e4a8990f9cea33c9fd7f2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:10.776597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:10.777782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedc984-7f6f-4a5a-a28c-30afba5dd4ad None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:10.778562 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 49/95] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:10 2022] GET /identity/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2 => generated 312 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:10.785757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:10.789810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 47/96] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:10 2022] GET /identity/v3/role_assignments?user.id=b0b3f354b8b6415a9dda8dd637e5335e&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 259 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:10.840759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:10.962183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea2693-0f13-44de-b5ce-774ba1475724 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:11.024879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:11.025537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:11.025826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:11.026107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:11.026938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=96f902ee515e4a8990f9cea33c9fd7f2, user.id=b0b3f354b8b6415a9dda8dd637e5335e) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:11.027256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:11.028962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d35eb-4a77-41ef-8e21-b42e14a43a9c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:11.036910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 50/97] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:10 2022] GET /identity/v3/role_assignments?user.id=b0b3f354b8b6415a9dda8dd637e5335e&scope.project.id=96f902ee515e4a8990f9cea33c9fd7f2&role.id=d47409cb4aa642018317ecedd3a36461 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:11.503134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2d4eb-11ba-43e0-9e6f-5e6b43da273d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:11.503685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2d4eb-11ba-43e0-9e6f-5e6b43da273d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:11.504155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2d4eb-11ba-43e0-9e6f-5e6b43da273d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:11.506365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 48/98] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:11.512694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:11.513056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:11.513440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:11.542913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:11.545035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:11.661951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 51/99] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:11.740698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2d4eb-11ba-43e0-9e6f-5e6b43da273d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:11.956698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2d4eb-11ba-43e0-9e6f-5e6b43da273d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:12.021319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d29c6-66d0-45a2-b413-2699b98afd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:12.023254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d29c6-66d0-45a2-b413-2699b98afd4c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.024151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d29c6-66d0-45a2-b413-2699b98afd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.025004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d29c6-66d0-45a2-b413-2699b98afd4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.055092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d29c6-66d0-45a2-b413-2699b98afd4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-536d29c6-66d0-45a2-b413-2699b98afd4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:12.066901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:12.068921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:12.070600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 49/100] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:12.078490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:12.116040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:12.143271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3906-6c37-4740-9054-2b90fc7ff4fc None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.143856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3906-6c37-4740-9054-2b90fc7ff4fc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.144389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3906-6c37-4740-9054-2b90fc7ff4fc None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.147341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 50/101] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:12.157809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.158346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.158820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.180140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:12.217362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:12.220292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:12.320326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9c91b-b21f-4da9-b2a5-df0c1be1bba8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:12.386992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:12.388066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.388531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.388994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.390367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:12.390888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:12.393644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:12.411078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 52/102] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:12.420499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:12.427649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 51/103] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:12.475248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:12.639074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:12.678054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396080-879b-45ef-98dc-60d1b8523d5c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:12.743689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:12.744678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.745140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.745609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] PATH_INFO: `/v3/users/a6db29fe3c0e42629b71a0ee201e268e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.774158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] RBAC: Authorizing `identity:get_user(user_id=a6db29fe3c0e42629b71a0ee201e268e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:12.776913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:12.780017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 53/104] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:12 2022] GET /identity/v3/users/a6db29fe3c0e42629b71a0ee201e268e => generated 289 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:12.793777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:12.816817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596b796-e193-4b2f-8670-23d14df43bcb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:12.883293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:12.901296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:12.901988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.902282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.902566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.908744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:12.909929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:12.911736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 52/105] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:12.918119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:12.920076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf6e7-61fa-45e1-ba93-e3ce1307291a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:12.963860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:12.964870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:12.965407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:12.965895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:12.979412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:13.051353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:13.053136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:13.162643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74093ad6-4359-4f6c-8deb-e9472c7944ae None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:13.191803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 54/106] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:13.198305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:13.254321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:13.255944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:13.256299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:13.256586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:13.257297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:13.259313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:13.302181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:13.303683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 53/107] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:05:12 2022] POST /identity/v3/projects => generated 302 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:05:13.392860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98f9f9-a1e2-4bed-9b78-709bc8311a24 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:13.433136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80ac0d-4211-4392-85c4-ffda2d2ce005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:13.433800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ac0d-4211-4392-85c4-ffda2d2ce005 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:13.434086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ac0d-4211-4392-85c4-ffda2d2ce005 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:13.434370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ac0d-4211-4392-85c4-ffda2d2ce005 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:13.435741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80ac0d-4211-4392-85c4-ffda2d2ce005 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:13.437899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80ac0d-4211-4392-85c4-ffda2d2ce005 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:13.440270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 55/108] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:13 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:13.447511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:13.480951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3a6c5-5c5a-4c9a-b097-5e5f25db5213 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:13.481281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3a6c5-5c5a-4c9a-b097-5e5f25db5213 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:13.481502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3a6c5-5c5a-4c9a-b097-5e5f25db5213 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:13.483048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 56/109] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:13.490653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:13.490922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:13.491230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:13.516326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:13.517567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:13.579594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:13.601209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 57/110] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:13.644072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:13.679011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cfc21-3088-4285-a21e-922fc30e4cf4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:13.693003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:13.767143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dec066-3222-4274-b5d3-f78cb72ea467 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:13.792692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:13.793751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:13.794245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:13.794743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:13.796849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] RBAC: Adding query filter params (role.id=4af4e31926f64e829ceca358f3b877f8, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=a6db29fe3c0e42629b71a0ee201e268e) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:13.797387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:13.800146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:13.809546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:13.810836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:13.811484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:13.812011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:13.819397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 54/111] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:13 2022] GET /identity/v3/role_assignments?user.id=a6db29fe3c0e42629b71a0ee201e268e&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=4af4e31926f64e829ceca358f3b877f8 => generated 259 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:13.841177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:13.854550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:13.856266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:13.857757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 58/112] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:13.868688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:14.101214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637614f-2754-47f1-9f01-02878ff72cf4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:14.135517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:14.166081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:14.167224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:14.167740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:14.168150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:14.169341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:14.169811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:14.172938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:14.197267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 55/113] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:14.204124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:14.388617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:14.398734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373c18f-dcbe-4cc1-abcc-f3b738ce5a16 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:14.465797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295f77-631b-461c-b761-b932ffd2d1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:14.466900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295f77-631b-461c-b761-b932ffd2d1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:14.467431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295f77-631b-461c-b761-b932ffd2d1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:14.467912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295f77-631b-461c-b761-b932ffd2d1f2 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:14.470381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295f77-631b-461c-b761-b932ffd2d1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:14.472849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295f77-631b-461c-b761-b932ffd2d1f2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:14.475993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 59/114] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:14 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:14.486043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:14.503583 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65713ad1-ffcb-42d7-b8c4-35303fee8c33 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:14.504037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65713ad1-ffcb-42d7-b8c4-35303fee8c33 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:14.504483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65713ad1-ffcb-42d7-b8c4-35303fee8c33 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:14.506775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 60/115] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:14.514488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:14.514951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:14.515325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:14.569524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:14.571613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:14.774306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde7559-9186-4a0b-83b1-3f50cd44996e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:14.794029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 61/116] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:14.825699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:14.826497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:14.826621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:14.826944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:14.828123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:14.829610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:14.831567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 56/117] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:14 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:14.837269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:14.850425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:14.951309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:15.003863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:15.016457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5181-a05f-4843-87cf-6c474524623c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:15.071066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bcf9f-98af-4138-8061-036998c220bf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:15.109570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456ec59-ae0f-4eb1-bf2b-255711217ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:15.110378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456ec59-ae0f-4eb1-bf2b-255711217ec8 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.110667 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456ec59-ae0f-4eb1-bf2b-255711217ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.111046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456ec59-ae0f-4eb1-bf2b-255711217ec8 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6/users/b0b3f354b8b6415a9dda8dd637e5335e/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.113196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456ec59-ae0f-4eb1-bf2b-255711217ec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e772eae3dbe343e896c3f799600481b6, user_id=b0b3f354b8b6415a9dda8dd637e5335e, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:15.114854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ff5a5-37bc-4adf-b774-c5adead993e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:15.114981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456ec59-ae0f-4eb1-bf2b-255711217ec8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:15.115662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ff5a5-37bc-4adf-b774-c5adead993e3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.116020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ff5a5-37bc-4adf-b774-c5adead993e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.116370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ff5a5-37bc-4adf-b774-c5adead993e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.131389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ff5a5-37bc-4adf-b774-c5adead993e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-a17ff5a5-37bc-4adf-b774-c5adead993e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:15.137357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:15.138372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:15.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:15.143816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 57/118] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:15.145473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd54bb-d72a-44cd-bbc2-a8922c85da0b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.145758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd54bb-d72a-44cd-bbc2-a8922c85da0b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.146043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd54bb-d72a-44cd-bbc2-a8922c85da0b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.147712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 58/119] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:15.149637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd54bb-d72a-44cd-bbc2-a8922c85da0b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:15.159088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 62/120] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:14 2022] PUT /identity/v3/projects/e772eae3dbe343e896c3f799600481b6/users/b0b3f354b8b6415a9dda8dd637e5335e/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:05:15.161485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.161789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.162082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.205092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:15.206271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:15.359694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 63/121] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:15.374811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd54bb-d72a-44cd-bbc2-a8922c85da0b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:15.384080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:15.422166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:15.423007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.423297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.423770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.424821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:15.425143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:15.426875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:15.427009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:15.448833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 59/122] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:15.465376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:15.625881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:15.667674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d917a-51ee-43c2-ad4d-791571e36796 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:15.681616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9166a87-10f2-4515-8efa-d146e5a9442c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:15.736432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:15.737055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.737348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.737630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.749297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12617e-a16a-47cc-af1d-241ccbc36245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:15.749496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:15.749991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12617e-a16a-47cc-af1d-241ccbc36245 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.750264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12617e-a16a-47cc-af1d-241ccbc36245 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.750543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12617e-a16a-47cc-af1d-241ccbc36245 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.751838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12617e-a16a-47cc-af1d-241ccbc36245 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:15.753313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12617e-a16a-47cc-af1d-241ccbc36245 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:15.755182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 60/123] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:15 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:15.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:15.756136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:15.756893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 64/124] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:15.762796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:15.764731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c033d5-804f-40fe-8044-a8c19d37e3a8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:15.765564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c033d5-804f-40fe-8044-a8c19d37e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:15.765851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c033d5-804f-40fe-8044-a8c19d37e3a8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:15.767243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 61/125] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:15.768028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c033d5-804f-40fe-8044-a8c19d37e3a8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:15.840293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:15.903519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57cdc7-2ab7-4f51-9a32-f4f7a0c2b589 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:15.962282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c033d5-804f-40fe-8044-a8c19d37e3a8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:16.010303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:16.011598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.012081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.012549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.013506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:16.014022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:16.016869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea772dd2-b1b8-42ed-b73b-92710383d3ff None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:16.018989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:16.020235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.020610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.021112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.028934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:16.031855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 62/126] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:16.032018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:16.034191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.034841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.035207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 65/127] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:15 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:16.035828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.041652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:16.085665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:16.086987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:16.088885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:16.262604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e64d92-faba-43e8-81f4-71391a1198f4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:16.286150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 63/128] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:16.287513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:16.329492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:16.330479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.330834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.331117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.335889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:16.338204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:16.342087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 66/129] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:16 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:16.348374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:16.382973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:16.520109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:16.528602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14deceb-cebe-4935-ac74-02924651c0d7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:16.569925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352aa2e-f66b-47f6-8a61-6aad00a22ecf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:16.583224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:16.584193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.584590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.584979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6/users/b0b3f354b8b6415a9dda8dd637e5335e/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.587479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] RBAC: Authorizing `identity:create_grant(project_id=e772eae3dbe343e896c3f799600481b6, user_id=b0b3f354b8b6415a9dda8dd637e5335e, role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:16.589205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:16.618618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 64/130] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:16 2022] PUT /identity/v3/projects/e772eae3dbe343e896c3f799600481b6/users/b0b3f354b8b6415a9dda8dd637e5335e/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:16.620354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:16.677645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:16.679531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.679955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.680421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.695057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:16.701723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:16.704018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:16.705532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 67/131] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:16.725570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:16.861553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a24fb-4732-4829-8163-85c4a8415602 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:16.905285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:16.906813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:16.911491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:16.911491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:16.911491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:16.911491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:16.911491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:16.922564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 68/132] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:16.937255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:16.938906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:16.943626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:16.966353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4391ac-2537-4a53-8f46-b1d3c9625950 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:17.029278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:17.032957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.034178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.034566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.037819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:17.040940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:17.047225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 65/133] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:16 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:17.053325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:17.196336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81e58d-5e40-4234-99d5-59f3b29a231b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:17.203947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed1a26-a038-4fc2-aa12-5e475261408f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:17.254579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddba06a-ab44-4903-9bfe-d076a217544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:17.256309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddba06a-ab44-4903-9bfe-d076a217544c None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.257179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddba06a-ab44-4903-9bfe-d076a217544c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.257879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddba06a-ab44-4903-9bfe-d076a217544c None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6/users/65c8d33b0a3b4ecb82a05c93cec78423/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.260757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddba06a-ab44-4903-9bfe-d076a217544c None admin] RBAC: Authorizing `identity:create_grant(project_id=e772eae3dbe343e896c3f799600481b6, user_id=65c8d33b0a3b4ecb82a05c93cec78423, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:17.264173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddba06a-ab44-4903-9bfe-d076a217544c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:17.276723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ac4f6-10a4-4dd8-963b-9c1f30f67fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:17.277808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ac4f6-10a4-4dd8-963b-9c1f30f67fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.278303 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ac4f6-10a4-4dd8-963b-9c1f30f67fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.278794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ac4f6-10a4-4dd8-963b-9c1f30f67fb4 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.281805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ac4f6-10a4-4dd8-963b-9c1f30f67fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:17.284578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ac4f6-10a4-4dd8-963b-9c1f30f67fb4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:17.286046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 66/134] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:17 2022] PUT /identity/v3/projects/e772eae3dbe343e896c3f799600481b6/users/65c8d33b0a3b4ecb82a05c93cec78423/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:17.287633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 69/135] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:16 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:17.288039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db715a40-0814-4842-8e55-8fe3f4348e77 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.293318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db715a40-0814-4842-8e55-8fe3f4348e77 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.293529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad28ee-c9a5-4c28-85dd-cfec02ea31fc None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.293529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad28ee-c9a5-4c28-85dd-cfec02ea31fc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.293529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad28ee-c9a5-4c28-85dd-cfec02ea31fc None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.293732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 70/136] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:17.294381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db715a40-0814-4842-8e55-8fe3f4348e77 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.295057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad28ee-c9a5-4c28-85dd-cfec02ea31fc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:17.297294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 67/137] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:17.303949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.304577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.305238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.348758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:17.350399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:17.377445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:17.400604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:17.447545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad28ee-c9a5-4c28-85dd-cfec02ea31fc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:17.501623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb89fde-7186-47f9-a88c-fb25fd63cc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:17.502300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89fde-7186-47f9-a88c-fb25fd63cc4b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.502580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89fde-7186-47f9-a88c-fb25fd63cc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.502853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89fde-7186-47f9-a88c-fb25fd63cc4b None admin] PATH_INFO: `/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.507484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb89fde-7186-47f9-a88c-fb25fd63cc4b None admin] RBAC: Authorizing `identity:get_project(project_id=96f902ee515e4a8990f9cea33c9fd7f2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:17.509347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb89fde-7186-47f9-a88c-fb25fd63cc4b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:17.511239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 71/138] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:17 2022] GET /identity/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:17.521368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.521856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.522333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.572780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 68/139] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:17.574099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:17.576411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:17.578544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:17.701379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 72/140] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:17.702432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:17.839031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:17.844128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224c70-338c-4e68-a644-3dd999e1bb8f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:17.880271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f666e9-ebcb-408b-962c-1e42d2f3a2ab None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:17.922979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:17.924057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.924543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.925013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.926449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:17.928181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:17.930458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 73/141] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:17.931759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:17.988348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:17.988983 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:17.989337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:17.989623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] PATH_INFO: `/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2/users/b0b3f354b8b6415a9dda8dd637e5335e/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:17.996460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] RBAC: Authorizing `identity:create_grant(project_id=96f902ee515e4a8990f9cea33c9fd7f2, user_id=b0b3f354b8b6415a9dda8dd637e5335e, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:17.997972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:18.017172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 69/142] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:17 2022] PUT /identity/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2/users/b0b3f354b8b6415a9dda8dd637e5335e/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:18.018258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:18.051499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:18.133943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae1b64-c55a-499c-a2e1-45ad9806c843 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:18.230454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093975eb-7d5a-4ef8-9235-160a0910d783 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:18.249951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:18.251240 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:18.251780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:18.252271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:18.264860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:18.278566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:18.284676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:18.285578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:18.286318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 74/143] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:18.293552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:18.336650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:18.337663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:18.338119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:18.338579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:18.339792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:18.342334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:18.396956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 70/144] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:18 2022] POST /identity/v3/users => generated 283 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:18.438237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57e35-84f8-4c8c-ad52-53137e7608c8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:18.482496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06294e-3f62-489e-8006-95066582045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:18.483148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06294e-3f62-489e-8006-95066582045e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:18.483982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06294e-3f62-489e-8006-95066582045e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:18.484366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06294e-3f62-489e-8006-95066582045e None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:18.485148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06294e-3f62-489e-8006-95066582045e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:18.485461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06294e-3f62-489e-8006-95066582045e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:18.486919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06294e-3f62-489e-8006-95066582045e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:18.499107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 75/145] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:18.508362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:18.631718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:18.743492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dbdfc-b28d-4608-bd03-89b7f41b1add None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:18.803631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:18.804667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:18.805189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:18.805733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] PATH_INFO: `/v3/users/a6db29fe3c0e42629b71a0ee201e268e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:18.807489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] RBAC: Authorizing `identity:get_user(user_id=a6db29fe3c0e42629b71a0ee201e268e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:18.808977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:18.810674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 71/146] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:18 2022] GET /identity/v3/users/a6db29fe3c0e42629b71a0ee201e268e => generated 289 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:18.817156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06294e-3f62-489e-8006-95066582045e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:18.909328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:18.983166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06294e-3f62-489e-8006-95066582045e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:19.034192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dd889-7d82-4f83-a552-5c3bb7652568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:19.034866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dd889-7d82-4f83-a552-5c3bb7652568 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:19.035146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dd889-7d82-4f83-a552-5c3bb7652568 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:19.035424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dd889-7d82-4f83-a552-5c3bb7652568 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:19.037110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dd889-7d82-4f83-a552-5c3bb7652568 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:19.038611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dd889-7d82-4f83-a552-5c3bb7652568 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:19.045729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 76/147] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:18 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:19.047351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:19.101844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00471e-3818-4581-acd1-90ab1b995d48 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:19.102125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00471e-3818-4581-acd1-90ab1b995d48 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:19.102317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00471e-3818-4581-acd1-90ab1b995d48 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:19.103905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 77/148] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:19.136559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:19.136956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:19.137167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:19.188687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:19.190084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:19.291942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db1abe-525f-48a0-97d7-850386ca3a3f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:19.380242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:19.382852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:19.384242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:19.386706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:19.387010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6/users/a6db29fe3c0e42629b71a0ee201e268e/roles/4af4e31926f64e829ceca358f3b877f8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:19.394194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] RBAC: Authorizing `identity:create_grant(project_id=e772eae3dbe343e896c3f799600481b6, user_id=a6db29fe3c0e42629b71a0ee201e268e, role_id=4af4e31926f64e829ceca358f3b877f8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:19.396819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:19.424885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 78/149] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:19.428440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 72/150] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:19 2022] PUT /identity/v3/projects/e772eae3dbe343e896c3f799600481b6/users/a6db29fe3c0e42629b71a0ee201e268e/roles/4af4e31926f64e829ceca358f3b877f8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:05:19.485382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:19.557512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:19.624713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72990014-4fae-45e4-8850-dc2075a9b947 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:19.671044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:19.671759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:19.672063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:19.672344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:19.688486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:19.697954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:19.699677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:19.701004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 79/151] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:19.707348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:19.870810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0e14-5b56-4264-ae2b-4bce7bc8761a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:19.914948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:19.915539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:19.915860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:19.916143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:19.916890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:19.917199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:19.918621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:19.928859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 73/152] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:19.937445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:20.095095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac78c97-e006-40b1-b5cc-0eabb3625c6d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:20.119935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:20.147343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7370eda-b1f5-4590-b40e-628f2300bca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:20.148079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7370eda-b1f5-4590-b40e-628f2300bca2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:20.148364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7370eda-b1f5-4590-b40e-628f2300bca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:20.148645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7370eda-b1f5-4590-b40e-628f2300bca2 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:20.152310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7370eda-b1f5-4590-b40e-628f2300bca2 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:20.153901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7370eda-b1f5-4590-b40e-628f2300bca2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:20.156062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 80/153] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:19 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:20.161957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:20.272054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:20.313578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ac892-bdbe-4e18-91a0-7b44a5b72f65 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:20.313921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ac892-bdbe-4e18-91a0-7b44a5b72f65 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:20.314145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ac892-bdbe-4e18-91a0-7b44a5b72f65 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:20.319610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 81/154] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:20.329003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:20.329333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:20.329567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:20.340421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923a057-f2d7-4197-be42-f962649ad411 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:20.372659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:20.373908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:20.385632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:20.386329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:20.386617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:20.386947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:20.474842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:20.476521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:20.513628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 82/155] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:20.568676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:20.576160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:20.650505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 74/156] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:20.668577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:20.819600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a84d9-2a80-460c-a7da-2d15c599d276 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:20.840520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:20.888840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:20.889504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:20.890321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:20.890577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:20.903608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e319e5d-92a5-4f1e-ba17-55724c49a649 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:20.913632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:20.918737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:20.919767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:20.920636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:20.927757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 83/157] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:20.933793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:20.977923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:20.978945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:20.979503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:20.980004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:20.984732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:20.986611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:20.988625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 75/158] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:20 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:20.995006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:21.050771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:21.105116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea63142-bee1-4dab-aee2-b3063f1cbdb4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:21.151544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:21.152549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:21.153032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:21.153518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:21.154828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:21.155900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:21.158383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:21.169642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 84/159] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:21.176027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:21.238763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:21.259223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a14d1-3b87-43e2-8ae4-53ff404b40b7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:21.332491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:21.333896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:21.334409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:21.334906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:21.336490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=b0b3f354b8b6415a9dda8dd637e5335e) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:21.337051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:21.339904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:21.353350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 76/160] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:20 2022] GET /identity/v3/role_assignments?user.id=b0b3f354b8b6415a9dda8dd637e5335e&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=d47409cb4aa642018317ecedd3a36461 => generated 608 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:21.470536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52dbcf-e807-412a-9a53-ea253dc3703c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:21.550256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d19a8a-4917-449a-8e1a-d481bd5017b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:21.551309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19a8a-4917-449a-8e1a-d481bd5017b6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:21.551810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19a8a-4917-449a-8e1a-d481bd5017b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:21.552272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19a8a-4917-449a-8e1a-d481bd5017b6 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:21.554430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d19a8a-4917-449a-8e1a-d481bd5017b6 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:21.557151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d19a8a-4917-449a-8e1a-d481bd5017b6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:21.560155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 85/161] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:21 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:21.569223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:21.763621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:21.784745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7d041-62de-4dd5-96c7-07327f97397c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:21.830283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a9053-049f-460e-b4a3-ab1e62644c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:21.831302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a9053-049f-460e-b4a3-ab1e62644c2f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:21.831820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a9053-049f-460e-b4a3-ab1e62644c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:21.832266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a9053-049f-460e-b4a3-ab1e62644c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:21.961611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a9053-049f-460e-b4a3-ab1e62644c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:21.964329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a9053-049f-460e-b4a3-ab1e62644c2f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:21.989033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da519b3-a272-470f-87b1-b1268542fee4 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:21.989378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da519b3-a272-470f-87b1-b1268542fee4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:21.989610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da519b3-a272-470f-87b1-b1268542fee4 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:21.991292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 86/162] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:21.996915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c2143-546c-4cc4-9db4-539bb17934f9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:21.997198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c2143-546c-4cc4-9db4-539bb17934f9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:21.997492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c2143-546c-4cc4-9db4-539bb17934f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:22.043915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-387c2143-546c-4cc4-9db4-539bb17934f9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:22.045055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-387c2143-546c-4cc4-9db4-539bb17934f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:22.150916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:22.185026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 77/163] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:22.187565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1045e0e-0bea-497d-8d85-6e6de5683772 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:22.188146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1045e0e-0bea-497d-8d85-6e6de5683772 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:22.188710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1045e0e-0bea-497d-8d85-6e6de5683772 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:22.191662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 78/164] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:22.197388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1045e0e-0bea-497d-8d85-6e6de5683772 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:22.206102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 87/165] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:22.213012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:22.213368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:22.213592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:22.240905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:22.242139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:22.286541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:22.381423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 88/166] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:22.382231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:22.474938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1045e0e-0bea-497d-8d85-6e6de5683772 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:22.570826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:22.571510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:22.571802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:22.572078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:22.573424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:22.574882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:22.576628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 79/167] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:22 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:22.577943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:22.655379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121ec3a-045b-4f90-8769-bfee52653420 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:22.730081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:22.753741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:22.754406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:22.754691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:22.754973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:22.781720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:22.793539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:22.795322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:22.796832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:22.802000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 89/168] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:22.804411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:22.865576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:22.915601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2ed2a-16b2-49de-8521-79ec8cab3ac6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.022582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6d10f-cf0b-45e1-87db-e8da6a41b400 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.028606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:23.035950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:23.036818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:23.037547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:23.059334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:23.066826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:23.067762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:23.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:23.070969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 80/169] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 494 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:23.072117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:23.140496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:23.141322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:23.141822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:23.142310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:23.143874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] RBAC: Adding query filter params (role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=b0b3f354b8b6415a9dda8dd637e5335e) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:23.144547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:23.147218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:23.167558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 90/170] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:22 2022] GET /identity/v3/role_assignments?user.id=b0b3f354b8b6415a9dda8dd637e5335e&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:23.169005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:23.227979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc1ba7-f61f-48c4-b765-d30c68f4a2b4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.246156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:23.274552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:23.275799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:23.276323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:23.276828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:23.277815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:23.278363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:23.280389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:23.290423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 81/171] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:23.298349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:23.399888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:23.445138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d834d-c0d7-4b5c-bea9-7ec7c5f757b8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.560133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:23.561278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:23.561820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:23.562316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:23.563720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:23.564280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:23.567013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:23.594227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 91/172] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:23.604016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:23.610680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4736e-fee7-442a-b180-271b76e35c73 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.739830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:23.740401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:23.740688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:23.740968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] PATH_INFO: `/v3/users/b0b3f354b8b6415a9dda8dd637e5335e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:23.742269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b3f354b8b6415a9dda8dd637e5335e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:23.743808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:23.745499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 82/173] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:23 2022] GET /identity/v3/users/b0b3f354b8b6415a9dda8dd637e5335e => generated 275 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:23.751817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:23.847943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:23.887302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb5bb6-cd4b-4aae-92c8-fc3e3244d487 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.986546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f43b1c-34b0-44d2-af68-b15318d0ca67 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:23.990634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacfc9a-ea3f-4a33-8c95-ee2bdeb9496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:23.990829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfc9a-ea3f-4a33-8c95-ee2bdeb9496f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:23.991113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfc9a-ea3f-4a33-8c95-ee2bdeb9496f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:23.991376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfc9a-ea3f-4a33-8c95-ee2bdeb9496f None admin] PATH_INFO: `/v3/users/65c8d33b0a3b4ecb82a05c93cec78423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:23.994237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfc9a-ea3f-4a33-8c95-ee2bdeb9496f None admin] RBAC: Authorizing `identity:get_user(user_id=65c8d33b0a3b4ecb82a05c93cec78423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:23.995233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfc9a-ea3f-4a33-8c95-ee2bdeb9496f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:23.997029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 92/174] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:23 2022] GET /identity/v3/users/65c8d33b0a3b4ecb82a05c93cec78423 => generated 247 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:24.007264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc9ab1-32ea-45f4-8993-ef804694e9dc None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.007264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc9ab1-32ea-45f4-8993-ef804694e9dc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.007264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc9ab1-32ea-45f4-8993-ef804694e9dc None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.007264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 93/175] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:24.008087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc9ab1-32ea-45f4-8993-ef804694e9dc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:24.053192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:24.066508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e49eef-af02-4e57-96c6-af11f0da5a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:24.068224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e49eef-af02-4e57-96c6-af11f0da5a4a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.069066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e49eef-af02-4e57-96c6-af11f0da5a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.069832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e49eef-af02-4e57-96c6-af11f0da5a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.169208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e49eef-af02-4e57-96c6-af11f0da5a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:24.171087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e49eef-af02-4e57-96c6-af11f0da5a4a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:24.219713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:24.232493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc9ab1-32ea-45f4-8993-ef804694e9dc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:24.280385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679cc6c-a087-44ef-8775-cb34f8586e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:24.281017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679cc6c-a087-44ef-8775-cb34f8586e85 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.281297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679cc6c-a087-44ef-8775-cb34f8586e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.281571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679cc6c-a087-44ef-8775-cb34f8586e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.287283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 83/176] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:24.289357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdba7ca-d47c-44f8-8a16-0600bde2e15b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.289864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdba7ca-d47c-44f8-8a16-0600bde2e15b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.290407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdba7ca-d47c-44f8-8a16-0600bde2e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.310311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2bdba7ca-d47c-44f8-8a16-0600bde2e15b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:24.313042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2bdba7ca-d47c-44f8-8a16-0600bde2e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:24.370036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679cc6c-a087-44ef-8775-cb34f8586e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:24.372046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679cc6c-a087-44ef-8775-cb34f8586e85 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:24.399324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:24.412529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 84/177] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:24.413969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c0d4-6d24-414a-bb28-77f69ec97908 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.414362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c0d4-6d24-414a-bb28-77f69ec97908 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.414750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c0d4-6d24-414a-bb28-77f69ec97908 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.416594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 85/178] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:24.422235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2c0d4-6d24-414a-bb28-77f69ec97908 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:24.521452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 94/179] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:24.526351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.526351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.526351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.546249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:24.548268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:24.549786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:24.607166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2c0d4-6d24-414a-bb28-77f69ec97908 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:24.646258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 95/180] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:24.646258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:24.675570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:24.676239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.676521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.676795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] PATH_INFO: `/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.678015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] RBAC: Authorizing `identity:get_project(project_id=96f902ee515e4a8990f9cea33c9fd7f2)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:24.679951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:24.686453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 86/181] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:24 2022] GET /identity/v3/projects/96f902ee515e4a8990f9cea33c9fd7f2 => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:24.686453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:24.725721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:24.786418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91c8b-2838-42c0-9ff8-13d0e648bcb2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:24.812616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cd00b-7e25-40c3-8389-de4b065a24ec None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:24.848332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:24.849600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.850033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.850474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.854159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2b91c-b1e4-4989-be3f-83af7d6ba582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:24.854806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2b91c-b1e4-4989-be3f-83af7d6ba582 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.855085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2b91c-b1e4-4989-be3f-83af7d6ba582 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.855388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2b91c-b1e4-4989-be3f-83af7d6ba582 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.856581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2b91c-b1e4-4989-be3f-83af7d6ba582 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:24.858051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2b91c-b1e4-4989-be3f-83af7d6ba582 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:24.860328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 87/182] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:24 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:24.861454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b934-ec8d-4772-934c-2f700aa62e3f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:24.861738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b934-ec8d-4772-934c-2f700aa62e3f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:24.862005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b934-ec8d-4772-934c-2f700aa62e3f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:24.863236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 88/183] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:24.863949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b934-ec8d-4772-934c-2f700aa62e3f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:24.872052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:24.876192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:24.877237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:24.878565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 96/184] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:24.879331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:24.978302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b934-ec8d-4772-934c-2f700aa62e3f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:24.988050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954920c1-a48e-40da-b17e-9351120015e8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:24.997321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:25.014795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7bcb0-fd81-4feb-8572-aa0972e2dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.015429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7bcb0-fd81-4feb-8572-aa0972e2dd2c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.015738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7bcb0-fd81-4feb-8572-aa0972e2dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.016011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7bcb0-fd81-4feb-8572-aa0972e2dd2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.024871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.025438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.025724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.026003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.026818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=96f902ee515e4a8990f9cea33c9fd7f2, user.id=b0b3f354b8b6415a9dda8dd637e5335e) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:25.026947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7bcb0-fd81-4feb-8572-aa0972e2dd2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:25.027128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.028644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb64f77-239c-43fa-937d-d90cdf3fe47d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-49f7bcb0-fd81-4feb-8572-aa0972e2dd2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:25.032542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:25.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:25.034254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 89/185] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:25.034996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01121cc3-c34e-4671-a98f-065cfbf0cf84 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.035575 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01121cc3-c34e-4671-a98f-065cfbf0cf84 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.035575 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01121cc3-c34e-4671-a98f-065cfbf0cf84 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.041644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 90/186] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:25.042121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 97/187] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:24 2022] GET /identity/v3/role_assignments?user.id=b0b3f354b8b6415a9dda8dd637e5335e&scope.project.id=96f902ee515e4a8990f9cea33c9fd7f2&role.id=d47409cb4aa642018317ecedd3a36461 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.042121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.042121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.042121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.042820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01121cc3-c34e-4671-a98f-065cfbf0cf84 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.060036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:25.061350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:25.133693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:25.156784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 98/188] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:25.157654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.192460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01121cc3-c34e-4671-a98f-065cfbf0cf84 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.251406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.252569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.253162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.253584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.254306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=65c8d33b0a3b4ecb82a05c93cec78423) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:25.254625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.258271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.266483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 91/189] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/role_assignments?user.id=65c8d33b0a3b4ecb82a05c93cec78423&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.267171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.304942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01880-da80-4d47-be30-1d93fe4eff03 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.307195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:25.369799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.370840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.371302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.371798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.373048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:25.373570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.376147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3208f6-cb40-4f0f-b550-671fffa9fdb1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.397075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 99/190] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.397075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.397075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.397075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.423589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:25.425784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:25.430335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda75f1-81e7-4ff2-8624-c6013e78ba09 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.486198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:25.492839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.493851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.494312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.494869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.495642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:25.495971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.497848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.513842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 92/191] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.516741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.544653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 100/192] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:25.545532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.663538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:25.668798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815fd26-0b43-4a0f-a687-5d8f698f0dd4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.676288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4d9bd-8e94-46bc-8dac-77f641660039 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.729402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.730260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.730503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.730779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.734935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.735779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.736163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.736524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] PATH_INFO: `/v3/users/15c2756f22ea44d585689c329eeb2209` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.743704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:25.749901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:25.751608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 93/193] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:25.752572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.757272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] RBAC: Authorizing `identity:get_user(user_id=15c2756f22ea44d585689c329eeb2209)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.758853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.760817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 101/194] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/users/15c2756f22ea44d585689c329eeb2209 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.761692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.824265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:25.875391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4431-01f4-4e61-96aa-89841c089515 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.901078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68438092-8f53-41d7-8803-432d61a384c6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:25.911818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.912527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.912752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.913027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] PATH_INFO: `/v3/users/a6db29fe3c0e42629b71a0ee201e268e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.914254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] RBAC: Authorizing `identity:get_user(user_id=a6db29fe3c0e42629b71a0ee201e268e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.916520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.918012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 94/195] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/users/a6db29fe3c0e42629b71a0ee201e268e => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.918012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.942824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:25.943505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:25.943795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:25.944075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:25.945496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:25.946887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:25.948564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 102/196] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:25.949340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:25.977251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.043883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e82e7-48ec-4c95-bf82-11a655cca981 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.071313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.085202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.085825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.086106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.086527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.086855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81b42-f1b9-4606-a38f-c78ee1f6fdc6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.087381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:26.087738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.089502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.099450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 95/197] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:26.101702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.128493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.129119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.129406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.129683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.209872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.211317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.222310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793c53d-3a0c-4deb-96ab-add204f16f41 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.236885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.268360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.269223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.269574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.269913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.329391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.336493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 103/198] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:26.337039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.355490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.358038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.444289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350c617-13d5-4bc7-81b8-ff259cd4bce8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.470642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.489477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.490307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.490660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.491006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.491900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.493880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.498073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 96/199] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:26.499903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.523344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 104/200] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:26 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:26.528689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.607222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.639066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fe525-0ab5-4efc-9b46-2d7560acb2d4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.685794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e2ccc-e049-4898-882b-9023952e4676 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.700659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.701490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.701943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.702877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] PATH_INFO: `/v3/users/15c2756f22ea44d585689c329eeb2209` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.707377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] RBAC: Authorizing `identity:get_user(user_id=15c2756f22ea44d585689c329eeb2209)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.710427 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.715984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 97/201] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:26 2022] GET /identity/v3/users/15c2756f22ea44d585689c329eeb2209 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:26.715984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.745100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.746157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.746722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.747264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.749878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.753162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.755568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 105/202] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:26 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:26.757046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.836282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:26.885855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96d70-e13a-454a-9f76-196d0f706a59 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.934870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e8d88-5ebc-459b-a7a9-49a93704aa8c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:26.941636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:26.942582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:26.943001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:26.943433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:26.945235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:26.947668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:26.950403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 98/203] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:26 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:26.953216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:26.990395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:27.009518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:27.010333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:27.010747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:27.011149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:27.090521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf9210-c776-4916-8c76-e512a73cde06 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:27.140217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:27.141370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:27.142241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:27.142530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:27.143724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=15c2756f22ea44d585689c329eeb2209) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:27.144067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:27.145706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:27.151233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:27.153239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:27.154029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 99/204] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:26 2022] GET /identity/v3/role_assignments?user.id=15c2756f22ea44d585689c329eeb2209&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:27.155642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:27.245497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:27.385202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:27.419710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 106/205] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:27.429422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:27.436562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce9386-bdec-4e79-a26d-c7fa9671c8da None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:27.552329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:27.552982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:27.553358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:27.554094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:27.554735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] RBAC: Adding query filter params (role.id=4af4e31926f64e829ceca358f3b877f8, scope.project.id=e772eae3dbe343e896c3f799600481b6, user.id=a6db29fe3c0e42629b71a0ee201e268e) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:27.555056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:27.556708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:27.561689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17b19-1968-4837-b843-a8e87b0804d4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:27.573923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 100/206] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:27 2022] GET /identity/v3/role_assignments?user.id=a6db29fe3c0e42629b71a0ee201e268e&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=4af4e31926f64e829ceca358f3b877f8 => generated 608 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:27.617622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfe6a2-6ab1-4c04-b0b5-5bfd700cce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:27.618649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe6a2-6ab1-4c04-b0b5-5bfd700cce2e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:27.619089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe6a2-6ab1-4c04-b0b5-5bfd700cce2e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:27.619539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe6a2-6ab1-4c04-b0b5-5bfd700cce2e None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:27.621179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfe6a2-6ab1-4c04-b0b5-5bfd700cce2e None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:27.623026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfe6a2-6ab1-4c04-b0b5-5bfd700cce2e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:27.626126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 107/207] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:27 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:27.632709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:27.847998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d22b5c-8061-45d4-a53b-3ef1f7352bb3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:27.910200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:27.911095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:27.911581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:27.912057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:27.913618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] RBAC: Adding query filter params (role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=15c2756f22ea44d585689c329eeb2209) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:27.914125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:27.916831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766d33-78af-44e6-81a5-b552d2701c5b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:27.926665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 101/208] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:27 2022] GET /identity/v3/role_assignments?user.id=15c2756f22ea44d585689c329eeb2209&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:29.259087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5060c-47c1-42a1-af43-278da756d22e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.259658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5060c-47c1-42a1-af43-278da756d22e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.259658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5060c-47c1-42a1-af43-278da756d22e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:29.261466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 108/209] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:29.267346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.267911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.267986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:29.288016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:29.289299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:29.298840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:29.372399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 102/210] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:29.396622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5060c-47c1-42a1-af43-278da756d22e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:29.418601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:29.522129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5060c-47c1-42a1-af43-278da756d22e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:29.563983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6e259-c9c6-4c1b-9a92-4603fa8a11e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:29.564661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e259-c9c6-4c1b-9a92-4603fa8a11e5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.564950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e259-c9c6-4c1b-9a92-4603fa8a11e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.565230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e259-c9c6-4c1b-9a92-4603fa8a11e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:29.566508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6e259-c9c6-4c1b-9a92-4603fa8a11e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:29.568102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6e259-c9c6-4c1b-9a92-4603fa8a11e5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:29.570242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 109/211] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:29.575954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:29.643022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb231-47e9-43e1-83aa-625f7df9ec8f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.643022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb231-47e9-43e1-83aa-625f7df9ec8f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.643022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb231-47e9-43e1-83aa-625f7df9ec8f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:29.645511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 110/212] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:29.652453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.652848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.653208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:29.675842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:29.677510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:29.728601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a6f49-f6d3-4445-9906-2209e3317370 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:29.762883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 111/213] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:29.777478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:29.778548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.779039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.779512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:29.780979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:29.782664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:29.788407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:29.805522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:29.807273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:29.823713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 103/214] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:29 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:29.941313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448fad4-f4c7-40fe-a2f2-0a96e1bf1149 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:29.989120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016afd75-79cb-4e26-848e-cad95981e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:29.989823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016afd75-79cb-4e26-848e-cad95981e5ae None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:29.990107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016afd75-79cb-4e26-848e-cad95981e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:29.990384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016afd75-79cb-4e26-848e-cad95981e5ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.004638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016afd75-79cb-4e26-848e-cad95981e5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-016afd75-79cb-4e26-848e-cad95981e5ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:30.009956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:30.010873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:30.011696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:30.016229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 112/215] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:30.022082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:30.078786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d23b6-5f16-45f7-85bb-4cebfe97e6bc None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.079519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d23b6-5f16-45f7-85bb-4cebfe97e6bc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.081704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d23b6-5f16-45f7-85bb-4cebfe97e6bc None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.084392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 113/216] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:30.096222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728f9e-3b61-4329-8c85-c8a770610fb2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.096765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728f9e-3b61-4329-8c85-c8a770610fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.097283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91728f9e-3b61-4329-8c85-c8a770610fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.144343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-91728f9e-3b61-4329-8c85-c8a770610fb2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:30.145642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-91728f9e-3b61-4329-8c85-c8a770610fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:30.188681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d00f8-05b9-4ea1-bed7-810ed2a10417 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:30.246460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:30.247127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.247411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.247728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.248504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:30.249389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:30.250881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:30.270337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 104/217] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:30.277622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:30.282092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:30.296443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 114/218] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:30.314565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e11cae-b790-4a39-88af-ef88fa48d576 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.315261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e11cae-b790-4a39-88af-ef88fa48d576 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.315995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e11cae-b790-4a39-88af-ef88fa48d576 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.318971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 115/219] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:30.328565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.329099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.329524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.361029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:30.362309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:30.395276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:30.463332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c8a9a-3433-4e7d-b112-a0a81245e27c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:30.464371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 116/220] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:30.472460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:30.519536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:30.520768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.521281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.521751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] PATH_INFO: `/v3/users/15c2756f22ea44d585689c329eeb2209` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.523553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] RBAC: Authorizing `identity:get_user(user_id=15c2756f22ea44d585689c329eeb2209)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:30.525995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:30.528655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 105/221] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:30 2022] GET /identity/v3/users/15c2756f22ea44d585689c329eeb2209 => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:30.534375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:30.651592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e162385-1d11-41f2-a6ad-bca9101d0796 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:30.701992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:30.703406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.704324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.704606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.707926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:30.719304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de6ce0-27d4-4772-936b-b36f5c060687 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:30.728544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:30.736484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:30.738442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:30.739677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:30.741720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 117/222] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:30.741839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:30.768919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:30.782358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:30.783655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:30.784244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:30.785000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:30.806473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:30.814940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:30.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 106/223] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:30.818109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:30.930997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c52e4-967f-428c-88fd-48991c2d1b5f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:30.992163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c40ec4-b7f7-4467-8463-255748287a98 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:30.998365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:30.999605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.000453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.001119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.002827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.004712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.006915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 118/224] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:30 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:31.012645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.043328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.044016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.044302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.044578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.047065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:31.047522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.049355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.062539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 107/225] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:31.067506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.102070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:31.133603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:31.170425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d835d-e08c-4ac4-90f7-82c4e19f1e0d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:31.206567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9d02-d7e5-4110-b687-f852b9bba35f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:31.215480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.216854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.217157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.217980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.218753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:31.219065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.221195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.234553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 119/226] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:31.235497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.255652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.256859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.257367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.257987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/15c2756f22ea44d585689c329eeb2209/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.260651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb545ff39f4342ab40fac4b4f6ad35, user_id=15c2756f22ea44d585689c329eeb2209, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.262750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.283666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 108/227] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:31 2022] PUT /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/15c2756f22ea44d585689c329eeb2209/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:31.290897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.475496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f64a2b-9b79-49bb-937c-2f35d1c9760f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:31.510987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:31.513552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065d5d-7347-4ab4-a387-ab8f72756035 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:31.526699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:31.541097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.542498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.543197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.543940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] PATH_INFO: `/v3/users/69e8749120184ac888e798eead720593` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.567098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=69e8749120184ac888e798eead720593)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.569946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.573062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 120/228] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:31 2022] GET /identity/v3/users/69e8749120184ac888e798eead720593 => generated 297 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:31.579859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.588628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.590013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.599572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.600344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] PATH_INFO: `/v3/users/15c2756f22ea44d585689c329eeb2209` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.602658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] RBAC: Authorizing `identity:get_user(user_id=15c2756f22ea44d585689c329eeb2209)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.605473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.612720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 109/229] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:31 2022] GET /identity/v3/users/15c2756f22ea44d585689c329eeb2209 => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:31.614365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.810516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5a9f3-0674-44eb-b010-b6cd8ea753a7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:31.836454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb56d4-f6f0-4e14-a419-ebeb8498d7b7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:31.855796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.856749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.857176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.857619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.861218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:31.861756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:31.867486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 110/230] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:31 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:31.869854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:31.919597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:31.920270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:31.922919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:31.922919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:31.961733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:32.052087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:32.087792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:32.090569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:32.094613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820752-3ea0-46f4-9caf-47eec095b35a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:32.208873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:32.209567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:32.209850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:32.210134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/15c2756f22ea44d585689c329eeb2209/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:32.221022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb545ff39f4342ab40fac4b4f6ad35, user_id=15c2756f22ea44d585689c329eeb2209, role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:32.222214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:32.245711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 121/231] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:05:32.248372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 111/232] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:31 2022] PUT /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/15c2756f22ea44d585689c329eeb2209/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:05:32.272685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:32.396015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:32.421832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83335a4-8221-4d7e-9486-afab37fd51ff None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:32.479946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b9bed-667c-4f2f-9b3c-cdfb5c4a5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:32.481480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b9bed-667c-4f2f-9b3c-cdfb5c4a5c36 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:32.482017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b9bed-667c-4f2f-9b3c-cdfb5c4a5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:32.483119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b9bed-667c-4f2f-9b3c-cdfb5c4a5c36 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:32.485647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b9bed-667c-4f2f-9b3c-cdfb5c4a5c36 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:32.487671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b9bed-667c-4f2f-9b3c-cdfb5c4a5c36 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:32.490362 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 122/233] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:32 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:32.505218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:32.719953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebc21d-4f93-4ebc-98df-d8fe7093ccc0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:32.734710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:32.776792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc08d-e223-4fe2-9b86-10b875315e84 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:32.777322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc08d-e223-4fe2-9b86-10b875315e84 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:32.777870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc08d-e223-4fe2-9b86-10b875315e84 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:32.780326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 123/234] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:32.788998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7c7a3-36ed-4ae8-af59-bf200f32710b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:32.789552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7c7a3-36ed-4ae8-af59-bf200f32710b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:32.790099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7c7a3-36ed-4ae8-af59-bf200f32710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:32.800074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:32.800861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:32.801149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:32.801432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:32.802452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=69e8749120184ac888e798eead720593) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:32.803166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:32.804352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aaa95-d3cd-4ac0-aefc-0befcbc743f8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:32.818120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 112/235] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:32 2022] GET /identity/v3/role_assignments?user.id=69e8749120184ac888e798eead720593&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=d47409cb4aa642018317ecedd3a36461 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:32.827063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9da7c7a3-36ed-4ae8-af59-bf200f32710b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:32.829008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9da7c7a3-36ed-4ae8-af59-bf200f32710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:33.036813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 124/236] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:33.067444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685b00-6372-4e35-805d-34793c5e2f2a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.067800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685b00-6372-4e35-805d-34793c5e2f2a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.068367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685b00-6372-4e35-805d-34793c5e2f2a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.070129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 113/237] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:33.077802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.078502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.079108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.086464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685b00-6372-4e35-805d-34793c5e2f2a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:33.116154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:33.118163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:33.271766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 125/238] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:33.311575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:33.317574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685b00-6372-4e35-805d-34793c5e2f2a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:33.329279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:33.389964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:33.391107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.391420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.394320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.426034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:33.440971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:33.442520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 114/239] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:33.449115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:33.518970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:33.588471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8a8b3-f9e2-4eb3-8f31-a9761e4efad1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:33.667443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54fe83-b8f4-4a12-a5e5-87c829c81ce0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:33.669889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:33.671094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.671384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.671700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.672993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:33.674733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:33.677171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 126/240] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:33.684116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:33.783354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:33.813158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:33.813842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.814127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.814404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.814762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51858d1-eda2-44f8-83de-293ef23e37ec None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:33.815142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:33.815468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:33.816913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:33.838722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 115/241] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:33.845182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:33.858739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07e3e6-810e-4f76-9280-9d0162e05ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:33.859380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e3e6-810e-4f76-9280-9d0162e05ba8 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.859696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e3e6-810e-4f76-9280-9d0162e05ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.859977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e3e6-810e-4f76-9280-9d0162e05ba8 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.860826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07e3e6-810e-4f76-9280-9d0162e05ba8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:33.862741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07e3e6-810e-4f76-9280-9d0162e05ba8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:33.896269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 127/242] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:33 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:05:33.949442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d88e61-8d97-4f5d-9441-7daff45d9413 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.949831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d88e61-8d97-4f5d-9441-7daff45d9413 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.950142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d88e61-8d97-4f5d-9441-7daff45d9413 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:33.956612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 128/243] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:33.958790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:33.959123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:33.959502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.025440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:34.027676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:34.038149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:34.051892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edfccd-b56c-4ca6-9240-21876ca6805f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:34.105181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:34.105941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:34.106307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:34.106645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] PATH_INFO: `/v3/users/e647e2bf4f6546b6959bd65848ea3724` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.129190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] RBAC: Authorizing `identity:get_user(user_id=e647e2bf4f6546b6959bd65848ea3724)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:34.131845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:34.135505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 116/244] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:33 2022] GET /identity/v3/users/e647e2bf4f6546b6959bd65848ea3724 => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:34.146862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.197328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 129/245] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:34.226772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.316690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:34.357079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979f2a-e2da-407f-b62b-8f260f82b41b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:34.367980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56dded-39ea-4f3c-8734-73f4d2c2bd77 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:34.425277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:34.426502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:34.426970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:34.427252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:34.427681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.428171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:34.428656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:34.429035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.444803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:34.451620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:34.452496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:34.453266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:34.456238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 130/246] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:34.462474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.465254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:34.542698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:34.545001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:34.608748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e18710-1329-4176-bba2-311cd7424d78 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:34.631826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:34.656482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:34.657534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:34.658066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:34.658545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.659871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:34.660511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:34.663134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:34.672416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 117/247] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:34.679332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.688830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 131/248] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:34.700633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.772598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:34.840001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617554b-fb15-4b1a-bef3-14188403a7ea None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:34.854025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224b769-dc56-4156-924d-dcebf841d7ee None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:34.889256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:34.890721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:34.891164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:34.891603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.892972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:34.894556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:34.894873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:34.895681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:34.896065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:34.896445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] PATH_INFO: `/v3/users/15c2756f22ea44d585689c329eeb2209` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:34.897120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 118/249] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:34.897923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] RBAC: Authorizing `identity:get_user(user_id=15c2756f22ea44d585689c329eeb2209)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:34.904340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:34.906123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.907032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 132/250] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/users/15c2756f22ea44d585689c329eeb2209 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:34.912959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:34.990999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:35.041570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6e6c6-4f58-4dd7-88d6-f5596ce0048a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:35.070633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:35.096984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871496c-6931-4e37-acff-96d3363403ed None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:35.117881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:35.119502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:35.119927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:35.120541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:35.121781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] RBAC: Adding query filter params (role.id=4af4e31926f64e829ceca358f3b877f8, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=e647e2bf4f6546b6959bd65848ea3724) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:35.122214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:35.124316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:35.143576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 119/251] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/role_assignments?user.id=e647e2bf4f6546b6959bd65848ea3724&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=4af4e31926f64e829ceca358f3b877f8 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:35.177150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:35.178573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:35.179264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:35.179916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:35.345749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:35.349244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:35.432012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:35.524629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 133/252] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:05:35.531289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:35.722500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15ac34-a3e5-4a5b-9fbc-2f8f0c826626 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:35.779803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275dda5-f579-422b-bf6a-25692e98ae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:35.781031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275dda5-f579-422b-bf6a-25692e98ae0e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:35.781313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275dda5-f579-422b-bf6a-25692e98ae0e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:35.781589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275dda5-f579-422b-bf6a-25692e98ae0e None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:35.788507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275dda5-f579-422b-bf6a-25692e98ae0e None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:35.790056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275dda5-f579-422b-bf6a-25692e98ae0e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:35.792496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 120/253] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:35 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:35.803495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:36.032607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd48171-0395-4840-a13f-f50dd0cec6fd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:36.100084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b7746-48bc-40e5-a17a-765b2f06b7ad None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.100743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:36.100913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b7746-48bc-40e5-a17a-765b2f06b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.101289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b7746-48bc-40e5-a17a-765b2f06b7ad None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.101548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.101813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.102102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.102916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=15c2756f22ea44d585689c329eeb2209) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:36.103230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:36.109041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:36.112011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 121/254] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:36 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:36.116127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 134/255] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:35 2022] GET /identity/v3/role_assignments?user.id=15c2756f22ea44d585689c329eeb2209&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:36.129915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43829efe-52da-4eb7-9d47-87ff8f5dde7d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.130223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43829efe-52da-4eb7-9d47-87ff8f5dde7d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.130535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43829efe-52da-4eb7-9d47-87ff8f5dde7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.165731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-43829efe-52da-4eb7-9d47-87ff8f5dde7d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:36.167018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-43829efe-52da-4eb7-9d47-87ff8f5dde7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:36.239049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:36.272100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 122/256] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:36.325046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:36.352752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:36.358470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15819-6f1d-4a33-bc37-98f2c40b3bc2 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.358828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15819-6f1d-4a33-bc37-98f2c40b3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.359063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15819-6f1d-4a33-bc37-98f2c40b3bc2 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.360785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 123/257] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:36.366269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.366912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.367213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.401935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:36.403240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:36.510138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ec9eb-6786-4a1d-8885-46fc0a7422d2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:36.570298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 124/258] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:36.590645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:36.591218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.591516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.591793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.608480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:36.614743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:36.617063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:36.619032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 135/259] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:36.627481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:36.634415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:36.829509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:36.881379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2f01a-1524-48cc-9f23-2823f44bd947 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:36.899693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369a1b5-1ec0-4c8d-a6c1-691ca3d3f1d7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:36.925065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:36.963529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:36.964688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.965192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.965749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.981249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:36.982212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:36.982681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:36.984614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:36.985579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:36.986064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:36.989898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:36.993356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:36.999477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:37.000644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:37.001725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:37.003907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 136/260] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:37.013406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:37.021899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 125/261] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:37.024804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:37.266504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001a99d-8d4e-42b3-ae72-06042b7bf948 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:37.272564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd1b72-f6e1-410b-a477-5064cec89bd1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:37.347416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:37.348107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.348409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.348746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] PATH_INFO: `/v3/users/15c2756f22ea44d585689c329eeb2209` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.350733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] RBAC: Authorizing `identity:get_user(user_id=15c2756f22ea44d585689c329eeb2209)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:37.354063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:37.355116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:37.355871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.356155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.356455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.357189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:37.357493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:37.359916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 126/262] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:37 2022] GET /identity/v3/users/15c2756f22ea44d585689c329eeb2209 => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:37.360785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:37.362430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:37.370921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:37.377096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 137/263] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:37.383155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:37.458718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:37.557244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8db18-16e3-4122-bafc-9d4cbd83eedc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:37.596840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:37.597492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.597810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.598030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] PATH_INFO: `/v3/users/69e8749120184ac888e798eead720593` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.599550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] RBAC: Authorizing `identity:get_user(user_id=69e8749120184ac888e798eead720593)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:37.601940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:37.603070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 138/264] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:37 2022] GET /identity/v3/users/69e8749120184ac888e798eead720593 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:37.612592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:37.625254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d5d76-6dfe-49e4-bdf2-d72f66a1c9dc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:37.677879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:37.678547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.678832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.679183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.710578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:37.765627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:37.767266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:37.769748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d3341-ae6f-40e4-820e-788988b92449 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:37.815681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af127bd-8e08-4080-8354-4c23b9afa365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:37.816364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af127bd-8e08-4080-8354-4c23b9afa365 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.816652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af127bd-8e08-4080-8354-4c23b9afa365 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.816935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af127bd-8e08-4080-8354-4c23b9afa365 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.818298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af127bd-8e08-4080-8354-4c23b9afa365 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:37.820439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af127bd-8e08-4080-8354-4c23b9afa365 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:37.822160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 139/265] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:37 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:37.824013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4341-8044-455b-8f4e-2230145904a0 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.824296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4341-8044-455b-8f4e-2230145904a0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.824572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4341-8044-455b-8f4e-2230145904a0 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.825937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 140/266] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:37.830940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:37.837566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:37.838091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:37.838091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:37.861881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:37.863098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:37.917558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 127/267] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:05:37.919322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:37.979149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 141/268] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:37.980434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.073571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:38.078457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90873d-71f3-46ec-9b1d-f4c0521a19e7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:38.119828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:38.121016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.121529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.122066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/69e8749120184ac888e798eead720593/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.124529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d83d9f-d508-4b82-abd2-fee6927942df None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:38.126503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb545ff39f4342ab40fac4b4f6ad35, user_id=69e8749120184ac888e798eead720593, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:38.129600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:38.147026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:38.156406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 128/269] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:37 2022] PUT /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/69e8749120184ac888e798eead720593/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:38.165594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.181553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:38.182621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.183101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.183638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.185345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:38.187879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:38.190233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 142/270] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:37 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:38.205441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210dd511-dd2c-4e17-a84c-1dd737f7a39e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:38.382522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f5413-5715-41fc-9756-f1522c50cd88 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:38.400890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bf920-5a3e-429e-8368-3acffa4a2ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:38.401801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bf920-5a3e-429e-8368-3acffa4a2ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.402294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bf920-5a3e-429e-8368-3acffa4a2ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.402576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bf920-5a3e-429e-8368-3acffa4a2ab3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.416294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bf920-5a3e-429e-8368-3acffa4a2ab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-188bf920-5a3e-429e-8368-3acffa4a2ab3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:38.422868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:38.424165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:38.425210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 129/271] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:38.426661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5c1a-cf56-44c4-b7d3-d90ba943e754 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.426661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5c1a-cf56-44c4-b7d3-d90ba943e754 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.426661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5c1a-cf56-44c4-b7d3-d90ba943e754 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.428563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 130/272] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:38.429504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777bbac-335f-4f65-8db4-a79771025bd2 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.429842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777bbac-335f-4f65-8db4-a79771025bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.430309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777bbac-335f-4f65-8db4-a79771025bd2 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.431843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 131/273] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:38.432631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777bbac-335f-4f65-8db4-a79771025bd2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.435039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:38.443974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:38.444618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.445017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.445333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.446185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] RBAC: Adding query filter params (role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=15c2756f22ea44d585689c329eeb2209) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:38.446652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:38.449619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea9f1-2d2c-43cd-a79e-16aaa8125fe9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:38.458612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 143/274] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:38 2022] GET /identity/v3/role_assignments?user.id=15c2756f22ea44d585689c329eeb2209&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:38.460178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19d07-17c8-4316-96fc-bc9dd4a21e0b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.460504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19d07-17c8-4316-96fc-bc9dd4a21e0b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.460717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e19d07-17c8-4316-96fc-bc9dd4a21e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.487589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-21e19d07-17c8-4316-96fc-bc9dd4a21e0b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:38.489271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-21e19d07-17c8-4316-96fc-bc9dd4a21e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:38.566204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:38.613982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 144/275] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:38.616874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.617800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.618541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.633101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777bbac-335f-4f65-8db4-a79771025bd2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:38.654688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:38.658093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:38.680630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:38.681304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.681593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.681944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.683018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:38.683216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:38.684859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:38.696159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 132/276] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:38.697735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.783563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 145/277] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:38.784821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.852847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4902dcf-cbd6-463f-8986-cfbfc3b9be92 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:38.880979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:38.896508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:38.921984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:38.922719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:38.922963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:38.923237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:38.942487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:38.948918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:38.949813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 133/278] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:38.956546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:38.980293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6a83-ea7e-4e29-8439-a14d6fb40137 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.026248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.026996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.027288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.027725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] PATH_INFO: `/v3/users/8cc3c347dc2846bd990887ee4fe7da3a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.047894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] RBAC: Authorizing `identity:get_user(user_id=8cc3c347dc2846bd990887ee4fe7da3a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.049801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.051987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 146/279] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:38 2022] GET /identity/v3/users/8cc3c347dc2846bd990887ee4fe7da3a => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:39.053063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:39.124059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362b072-e603-4780-ba5b-0339c23a9aab None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.185231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.186211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.186663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.187100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.189138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.191782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.194853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 134/280] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:39.196336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:39.218613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0136-7cd1-4284-bff5-640b5cbe6d6c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.245170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:39.286351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.287140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.287425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.287731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.288509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:39.289312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.290859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:39.292195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.306319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 147/281] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:39.308999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:39.370249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbf4a7-c518-4a35-8de0-88ae95c16082 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.425702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.426725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.427207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.427799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.429278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, user.id=8cc3c347dc2846bd990887ee4fe7da3a, scope.system=all) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:39.429987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.431848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.438375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 135/282] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Aug 1 14:05:39 2022] GET /identity/v3/role_assignments?user.id=8cc3c347dc2846bd990887ee4fe7da3a&scope.system=all&role.id=d47409cb4aa642018317ecedd3a36461 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:39.440009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:39.478476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979f8ed-d08c-477f-8390-3dd59ec99ab8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.541211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.541870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.542156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.542437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.543170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.544812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.580206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 148/283] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:05:39 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:39.652816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc501a7-ffe4-4249-9847-422a0ad9e575 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.684109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:39.720254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.721226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.721715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.722156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] PATH_INFO: `/v3/users/e647e2bf4f6546b6959bd65848ea3724` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.724627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] RBAC: Authorizing `identity:get_user(user_id=e647e2bf4f6546b6959bd65848ea3724)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.728032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.731844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 136/284] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:39 2022] GET /identity/v3/users/e647e2bf4f6546b6959bd65848ea3724 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:39.740428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:39.789281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:39.897623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c58046-ffba-4f6f-ad2e-9a5f4a7a3f3b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:39.961451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62db95-4ebd-46a2-8cd9-ed972f17af7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:39.962808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62db95-4ebd-46a2-8cd9-ed972f17af7f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:39.963624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62db95-4ebd-46a2-8cd9-ed972f17af7f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:39.964482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62db95-4ebd-46a2-8cd9-ed972f17af7f None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:39.967078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62db95-4ebd-46a2-8cd9-ed972f17af7f None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:39.969115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62db95-4ebd-46a2-8cd9-ed972f17af7f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:39.971780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 149/285] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:39 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:39.978694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:40.127841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a52833-3f1f-4afa-ba2e-b0b7b09bcf21 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:40.169855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acd2c5-0a01-4fbb-9d5f-4b7ba980d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:40.171352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acd2c5-0a01-4fbb-9d5f-4b7ba980d4b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:40.172112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acd2c5-0a01-4fbb-9d5f-4b7ba980d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:40.172816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acd2c5-0a01-4fbb-9d5f-4b7ba980d4b6 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/e647e2bf4f6546b6959bd65848ea3724/roles/4af4e31926f64e829ceca358f3b877f8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:40.176734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acd2c5-0a01-4fbb-9d5f-4b7ba980d4b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb545ff39f4342ab40fac4b4f6ad35, user_id=e647e2bf4f6546b6959bd65848ea3724, role_id=4af4e31926f64e829ceca358f3b877f8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:40.179809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acd2c5-0a01-4fbb-9d5f-4b7ba980d4b6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:40.207157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 137/286] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:05:39 2022] PUT /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/users/e647e2bf4f6546b6959bd65848ea3724/roles/4af4e31926f64e829ceca358f3b877f8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:40.828236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c45bf-5d35-4f80-be23-c28c4a635dc2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:40.829278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c45bf-5d35-4f80-be23-c28c4a635dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:40.830112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c45bf-5d35-4f80-be23-c28c4a635dc2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:40.833413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 150/287] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:40.842907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:40.843615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:40.844114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:40.883397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:40.885935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:40.949250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:41.000599 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 138/288] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:41.036452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c45bf-5d35-4f80-be23-c28c4a635dc2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:41.191616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:41.199376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c45bf-5d35-4f80-be23-c28c4a635dc2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:41.284041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:41.285250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:41.285732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:41.286195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:41.303702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:41.314707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:41.315749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:41.316664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:41.317403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 151/289] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:41.330964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:41.549571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca9f06-5655-4093-9dcb-18be3f8648d0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:41.606551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:41.606551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:41.607035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:41.607422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:41.610557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:41.610557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:41.612432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:41.635605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 139/290] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:41.641355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:41.829548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10bedd-7767-4aaa-bb6d-4161041dba12 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:41.916625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:41.917725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:41.918214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:41.918702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] PATH_INFO: `/v3/users/69e8749120184ac888e798eead720593` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:41.920897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] RBAC: Authorizing `identity:get_user(user_id=69e8749120184ac888e798eead720593)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:41.923149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:41.930393 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 152/291] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:41 2022] GET /identity/v3/users/69e8749120184ac888e798eead720593 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:41.936154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:42.023164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:42.144542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a7ffb-db6e-4398-a02d-298b898c40ee None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:42.193039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eebba-434c-4ba6-9c19-30e902df7895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:42.193772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eebba-434c-4ba6-9c19-30e902df7895 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.194058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eebba-434c-4ba6-9c19-30e902df7895 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.194338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eebba-434c-4ba6-9c19-30e902df7895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.306993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eebba-434c-4ba6-9c19-30e902df7895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:42.308573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eebba-434c-4ba6-9c19-30e902df7895 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:42.374170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:42.455069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 140/292] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:05:42.461421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:42.519829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:42.601054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cd7d7-ef4b-4699-a1b8-4b8eb6b47d8b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.601482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cd7d7-ef4b-4699-a1b8-4b8eb6b47d8b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.601900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cd7d7-ef4b-4699-a1b8-4b8eb6b47d8b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.608187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 141/293] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:42.613265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf4bd-cbae-4aa9-ae35-f0852cda0dc9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.613678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf4bd-cbae-4aa9-ae35-f0852cda0dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.614085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf4bd-cbae-4aa9-ae35-f0852cda0dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.647926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-57dcf4bd-cbae-4aa9-ae35-f0852cda0dc9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:42.647926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-57dcf4bd-cbae-4aa9-ae35-f0852cda0dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:42.665196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d0c06-92bc-4f4a-bc4a-443d759bc806 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:42.735672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:42.736238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.736525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.736859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.739700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:42.741399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:42.753293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 153/294] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:42 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:42.757636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:42.782418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 142/295] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:42.783807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54ab4-8072-45c4-83b8-756371826acc None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.784091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54ab4-8072-45c4-83b8-756371826acc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.784364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54ab4-8072-45c4-83b8-756371826acc None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.786443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 143/296] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:42.792187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a57e7-f6b3-40c5-9072-38837ce80fee None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.792469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a57e7-f6b3-40c5-9072-38837ce80fee None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.792747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a57e7-f6b3-40c5-9072-38837ce80fee None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.799115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 144/297] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:42.799115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371ae7-45dc-4fee-be03-0f13c87b4477 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.799115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371ae7-45dc-4fee-be03-0f13c87b4477 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.799115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371ae7-45dc-4fee-be03-0f13c87b4477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.817361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1d371ae7-45dc-4fee-be03-0f13c87b4477 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:42.818568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1d371ae7-45dc-4fee-be03-0f13c87b4477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:42.880171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:42.891534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6aca-16c1-4901-b1a8-36b3936aa6e2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:42.892846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:42.903569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 145/298] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:42.905507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.905794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.906069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.926525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:42.927752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:42.941070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:42.942053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:42.942625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:42.943191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:42.944450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=69e8749120184ac888e798eead720593) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:42.945054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:42.946809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:42.953946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 154/299] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:42 2022] GET /identity/v3/role_assignments?user.id=69e8749120184ac888e798eead720593&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=d47409cb4aa642018317ecedd3a36461 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:42.954954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.033764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 146/300] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:43.035769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.156399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107b3a7-b525-4f7c-8100-7207d4c5fafd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.201771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.202354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.202640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.202919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.214881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:43.221200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:43.222418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 155/301] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:43.223389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.237774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fced0a-f127-43cb-a933-dda4c6610428 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.254178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:43.270839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:43.284132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebe7e5-c33e-449a-8bb7-9f8c77f3ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.284813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe7e5-c33e-449a-8bb7-9f8c77f3ecaf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.285099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe7e5-c33e-449a-8bb7-9f8c77f3ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.285374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe7e5-c33e-449a-8bb7-9f8c77f3ecaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.286539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebe7e5-c33e-449a-8bb7-9f8c77f3ecaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:43.288058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebe7e5-c33e-449a-8bb7-9f8c77f3ecaf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:43.289683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 147/302] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:43.292978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3400cd-719d-42ca-90f9-6747aba4497c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.293412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3400cd-719d-42ca-90f9-6747aba4497c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.293876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3400cd-719d-42ca-90f9-6747aba4497c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.295764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 148/303] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:43.296548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3400cd-719d-42ca-90f9-6747aba4497c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.356758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153f572-5f59-4a61-a1d7-c2fb52512624 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.398187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.398935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.399221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.399563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.412956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:43.420018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:43.421999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 156/304] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:43.423679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.433757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3400cd-719d-42ca-90f9-6747aba4497c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.479278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.479931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.480213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.480499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.481218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:43.481520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:43.482927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b2edc-f433-40b5-837f-eaf04c42323f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:43.496579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 149/305] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:43.496579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.496579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.496579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.518172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:43.519418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:43.521049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:43.545152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e8151-a328-4407-8621-9411f4d1e72c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.583739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.584373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.584653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.584930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] PATH_INFO: `/v3/groups` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.585627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:43.586847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:43.587191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:43.606193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 157/306] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Aug 1 14:05:43 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:43.607046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.610730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 150/307] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:43.611894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.789348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc7053-7a91-4dc3-8db5-0022c7cef1dc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.812008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332e8bf-7b55-4821-a9eb-34a4a764342c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:43.851683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.852435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.852744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.853108 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] PATH_INFO: `/v3/users/8cc3c347dc2846bd990887ee4fe7da3a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.854342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] RBAC: Authorizing `identity:get_user(user_id=8cc3c347dc2846bd990887ee4fe7da3a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:43.855854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:43.858039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 151/308] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:43 2022] GET /identity/v3/users/8cc3c347dc2846bd990887ee4fe7da3a => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:43.858692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.872989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:43.874293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:43.874581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:43.874851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:43.875615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:43.875921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:43.877448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:43.885028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:43.890922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 158/309] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:43.896061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:43.926973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:44.003334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9147e-bf3b-48ae-b0b9-8b77ec14385a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.023723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20ad97-7ddc-4e24-b143-3e9284be8653 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.047299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.048368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.048840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.049304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.067134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:44.076003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:44.078005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 152/310] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:44.079614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:44.086017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.087041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.087433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.087890 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] PATH_INFO: `/v3/system/users/8cc3c347dc2846bd990887ee4fe7da3a/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.089590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8cc3c347dc2846bd990887ee4fe7da3a, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:44.091121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:44.111714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 159/311] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Aug 1 14:05:43 2022] PUT /identity/v3/system/users/8cc3c347dc2846bd990887ee4fe7da3a/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:44.111714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:44.245402 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:44.256103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc8dab-920b-410c-9247-26b09aafce05 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.275579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:44.293378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70e5b5-28d9-411f-94cd-02096a9a5b48 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.314679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.315677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.316297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.316719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] PATH_INFO: `/v3/users/6c97bf50f080451d8f6f4919e6e059f4` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.337408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] RBAC: Authorizing `identity:get_user(user_id=6c97bf50f080451d8f6f4919e6e059f4)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:44.339421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:44.341559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 153/312] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:44 2022] GET /identity/v3/users/6c97bf50f080451d8f6f4919e6e059f4 => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:44.348355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:44.350570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.351271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.352485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.352485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.353776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:44.353776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:44.355995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:44.372175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 160/313] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:44.379690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:44.506757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e73c9-1a6c-4bce-a56f-defce18daa7a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.554342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd15862-c1a2-4924-9df0-75487c452cdf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.573495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.575058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.575862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.577160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.578876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] RBAC: Adding query filter params (role.id=4af4e31926f64e829ceca358f3b877f8, user.id=6c97bf50f080451d8f6f4919e6e059f4, scope.system=all) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:44.579802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:44.582742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:44.596004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 154/314] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Aug 1 14:05:44 2022] GET /identity/v3/role_assignments?user.id=6c97bf50f080451d8f6f4919e6e059f4&scope.system=all&role.id=4af4e31926f64e829ceca358f3b877f8 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:44.611218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:44.628832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.629905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.630364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.630805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] PATH_INFO: `/v3/users/e647e2bf4f6546b6959bd65848ea3724` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.633144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] RBAC: Authorizing `identity:get_user(user_id=e647e2bf4f6546b6959bd65848ea3724)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:44.635918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:44.651814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 161/315] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:44 2022] GET /identity/v3/users/e647e2bf4f6546b6959bd65848ea3724 => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:44.653542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:44.726650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:44.818943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46839f3c-54a6-486e-93ae-7a78d36e277e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:44.866738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:44.867799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:44.868657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:44.869152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:44.976237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:44.979155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:45.047826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:45.139714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 155/316] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:45.141766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:45.307148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec7d76-6dc6-4271-837f-449d68dda49c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:45.348363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df582e-4948-4428-b7bb-5155c77cfdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:45.349359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df582e-4948-4428-b7bb-5155c77cfdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:45.349646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df582e-4948-4428-b7bb-5155c77cfdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:45.349923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df582e-4948-4428-b7bb-5155c77cfdc4 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:45.351300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df582e-4948-4428-b7bb-5155c77cfdc4 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:45.352838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df582e-4948-4428-b7bb-5155c77cfdc4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:45.355066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 162/317] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:45 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:45.365131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:45.506033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7ff4-ab74-41fa-be33-c742ae1d3d16 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:45.569014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:45.569679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:45.569962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:45.570236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:45.571064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] RBAC: Adding query filter params (role.id=4af4e31926f64e829ceca358f3b877f8, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35, user.id=e647e2bf4f6546b6959bd65848ea3724) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:45.571379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:45.572918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2f4b-bb54-425f-a15f-01ff80a1de63 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:45.575215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:45.581286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 156/318] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:05:45 2022] GET /identity/v3/role_assignments?user.id=e647e2bf4f6546b6959bd65848ea3724&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=4af4e31926f64e829ceca358f3b877f8 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:45.969154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109c16-c4c8-4f10-b639-bed0de994ecf None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:45.969876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109c16-c4c8-4f10-b639-bed0de994ecf None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:45.969876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109c16-c4c8-4f10-b639-bed0de994ecf None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:45.971262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 163/319] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:45.977776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:45.978171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:45.978634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:46.000266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:46.001596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:46.112920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 157/320] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:46.138726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109c16-c4c8-4f10-b639-bed0de994ecf None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:46.222963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:46.289182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109c16-c4c8-4f10-b639-bed0de994ecf None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:46.350400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5ac6f-23a8-408d-b0df-d210fc4e8a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:46.351089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5ac6f-23a8-408d-b0df-d210fc4e8a77 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:46.351392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5ac6f-23a8-408d-b0df-d210fc4e8a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:46.351699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5ac6f-23a8-408d-b0df-d210fc4e8a77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:46.353027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5ac6f-23a8-408d-b0df-d210fc4e8a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:46.355716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5ac6f-23a8-408d-b0df-d210fc4e8a77 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:46.358330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 164/321] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:05:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:46.364200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:46.510335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543621e-75d6-47cc-b9a9-53c6d6ac1f6b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:46.542296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:46.570437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2ab5c-0ebd-43c4-a7d3-f8eea9fdc60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:46.571517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2ab5c-0ebd-43c4-a7d3-f8eea9fdc60e None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:46.572024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2ab5c-0ebd-43c4-a7d3-f8eea9fdc60e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:46.572516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2ab5c-0ebd-43c4-a7d3-f8eea9fdc60e None admin] PATH_INFO: `/v3/groups` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:46.573713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2ab5c-0ebd-43c4-a7d3-f8eea9fdc60e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:46.576512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2ab5c-0ebd-43c4-a7d3-f8eea9fdc60e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:46.602928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 158/322] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Aug 1 14:05:46 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:46.807418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111d758-190a-4454-8491-c8146cdbde3c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:46.807700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111d758-190a-4454-8491-c8146cdbde3c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:46.807925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111d758-190a-4454-8491-c8146cdbde3c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:46.809484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 165/323] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:46.817432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037f9ce-e015-46f5-9da7-8f47bd89121b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:46.817818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037f9ce-e015-46f5-9da7-8f47bd89121b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:46.818182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037f9ce-e015-46f5-9da7-8f47bd89121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:46.839598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5037f9ce-e015-46f5-9da7-8f47bd89121b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:46.840851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5037f9ce-e015-46f5-9da7-8f47bd89121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:46.936550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 159/324] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:46.962632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111d758-190a-4454-8491-c8146cdbde3c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:47.129676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30ff7f-7a0e-4652-b768-9a310be1e2ff None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.130888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30ff7f-7a0e-4652-b768-9a310be1e2ff None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.131422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30ff7f-7a0e-4652-b768-9a310be1e2ff None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.137195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111d758-190a-4454-8491-c8146cdbde3c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:47.138536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 160/325] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:05:47.145546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.146312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.146999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.178073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:47.181623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:47.184464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:47.198203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:47.202382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.202845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.203134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.222423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:47.233846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:47.234852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:47.236320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 166/326] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:47.243784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:47.339834 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 161/327] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:47.393292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ef70d-d7ee-4ce4-a340-bed9a1c6eeb5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:47.395370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:47.440656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:47.441296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.441584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.441859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.442605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:47.442916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:47.444481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:47.455258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 167/328] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:47.465299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:47.507854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:47.568883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:47.594090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffccab1-8038-4f7c-b1f9-f358fcce955d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:47.642847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:47.642847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.642847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.642847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.650303 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35da49-8a26-41fd-82b8-85c40b4fa65f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:47.660251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:47.666738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:47.667726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:47.668507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 162/329] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:47.674778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:47.712703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:47.713799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.714265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.715017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] PATH_INFO: `/v3/users/8cc3c347dc2846bd990887ee4fe7da3a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.716870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] RBAC: Authorizing `identity:get_user(user_id=8cc3c347dc2846bd990887ee4fe7da3a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:47.719667 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:47.722510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 168/330] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:47 2022] GET /identity/v3/users/8cc3c347dc2846bd990887ee4fe7da3a => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:47.733615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:47.890501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86a7b-8523-477c-9988-40f67f486c5d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:47.919747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:47.925662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38634dae-4b65-4ce5-bbf1-655b1a7ebd98 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:47.941129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:47.942121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:47.944655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:47.945222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:47.946670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] RBAC: Adding query filter params (role.id=d47409cb4aa642018317ecedd3a36461, user.id=8cc3c347dc2846bd990887ee4fe7da3a, scope.system=all) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:47.947403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:47.948297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:47.950368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:47.968448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 169/331] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Aug 1 14:05:47 2022] GET /identity/v3/role_assignments?user.id=8cc3c347dc2846bd990887ee4fe7da3a&scope.system=all&role.id=d47409cb4aa642018317ecedd3a36461 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:48.012043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:48.012750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:48.013137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:48.013518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:48.014372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:48.015034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:48.016497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:48.027407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 163/332] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:48.034437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:48.223216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3b18-7998-4f34-92b6-f14cf22f5709 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:48.298602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dbfd2-ea66-4cec-827f-3d36620f5d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:48.300243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dbfd2-ea66-4cec-827f-3d36620f5d69 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:48.301011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dbfd2-ea66-4cec-827f-3d36620f5d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:48.301495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dbfd2-ea66-4cec-827f-3d36620f5d69 None admin] PATH_INFO: `/v3/users/6c97bf50f080451d8f6f4919e6e059f4` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:48.308455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dbfd2-ea66-4cec-827f-3d36620f5d69 None admin] RBAC: Authorizing `identity:get_user(user_id=6c97bf50f080451d8f6f4919e6e059f4)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:48.310304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dbfd2-ea66-4cec-827f-3d36620f5d69 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:48.315170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 170/333] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:48 2022] GET /identity/v3/users/6c97bf50f080451d8f6f4919e6e059f4 => generated 293 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:48.324714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:48.546986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec845d9-3f68-4ca5-8c72-caa8939eb0f9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:48.617998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ec5aa-3d78-48da-8ccf-2550a49ffac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:48.618706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec5aa-3d78-48da-8ccf-2550a49ffac9 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:48.618991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec5aa-3d78-48da-8ccf-2550a49ffac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:48.619269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec5aa-3d78-48da-8ccf-2550a49ffac9 None admin] PATH_INFO: `/v3/system/users/6c97bf50f080451d8f6f4919e6e059f4/roles/4af4e31926f64e829ceca358f3b877f8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:48.622734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ec5aa-3d78-48da-8ccf-2550a49ffac9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6c97bf50f080451d8f6f4919e6e059f4, role_id=4af4e31926f64e829ceca358f3b877f8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:48.624413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ec5aa-3d78-48da-8ccf-2550a49ffac9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:48.639761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 164/334] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Aug 1 14:05:48 2022] PUT /identity/v3/system/users/6c97bf50f080451d8f6f4919e6e059f4/roles/4af4e31926f64e829ceca358f3b877f8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:50.519004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079adeb-643b-4321-920d-02d261bca4e1 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:50.520084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079adeb-643b-4321-920d-02d261bca4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:50.520586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079adeb-643b-4321-920d-02d261bca4e1 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:50.522195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 171/335] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:50.531381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66e58-cd6c-42a4-8249-58f1b3b33b2d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:50.532373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66e58-cd6c-42a4-8249-58f1b3b33b2d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:50.532861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66e58-cd6c-42a4-8249-58f1b3b33b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:50.557232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-03b66e58-cd6c-42a4-8249-58f1b3b33b2d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:50.558817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-03b66e58-cd6c-42a4-8249-58f1b3b33b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:50.603841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:50.670466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 165/336] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:50.706243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079adeb-643b-4321-920d-02d261bca4e1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:50.776390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:50.800126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c4ce0-5caa-4470-a16f-d766c0a46367 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:50.800630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c4ce0-5caa-4470-a16f-d766c0a46367 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:50.801127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c4ce0-5caa-4470-a16f-d766c0a46367 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:50.803232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 166/337] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:50.813536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:50.813910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:50.814174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:50.844197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:50.845541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:50.910034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079adeb-643b-4321-920d-02d261bca4e1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:50.944399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 167/338] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:50.972267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:50.998863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:50.999870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.000310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.000736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.034266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:51.048508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:51.049462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:51.051498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 172/339] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:51.064042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:51.194553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:51.213920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3aced-6eb6-4d13-90a2-a96b56d02785 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:51.295711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca9b2b-81bb-4749-885e-a6aff304e941 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:51.303791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:51.305616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dc595-0009-437b-b1a7-389fe18b84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:51.306371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dc595-0009-437b-b1a7-389fe18b84a9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.306705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dc595-0009-437b-b1a7-389fe18b84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.306977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dc595-0009-437b-b1a7-389fe18b84a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.336494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825dc595-0009-437b-b1a7-389fe18b84a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:51.341161 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:51.342339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.342624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-825dc595-0009-437b-b1a7-389fe18b84a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:51.342858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:51.344258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:51.346062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:51.347069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.347069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:51.347069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:51.347370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 168/340] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:51.347498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539096b-5f59-4815-929e-a43ed49a30c3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:51.347622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52d7a0-b13d-416e-afb8-3379ab86b182 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.347622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52d7a0-b13d-416e-afb8-3379ab86b182 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.347622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52d7a0-b13d-416e-afb8-3379ab86b182 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.353550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 169/341] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:51.354302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52d7a0-b13d-416e-afb8-3379ab86b182 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:51.362815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 173/342] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:51.368206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.368515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.368770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.388054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:51.389357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:51.466500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 174/343] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:51.467686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:51.577304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52d7a0-b13d-416e-afb8-3379ab86b182 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:51.577963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:51.587922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763ff85-397a-4c46-aa6e-e983f9ed1b28 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:51.620814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:51.621454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.621769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.622011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.622745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:51.623191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:51.624700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae8dd6-62a2-4137-a4b8-317e1eb2e4f4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:51.638379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 170/344] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:51.640966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b5770-2b1a-4bdb-bd37-65e3a2b2072b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.640966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b5770-2b1a-4bdb-bd37-65e3a2b2072b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.640966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b5770-2b1a-4bdb-bd37-65e3a2b2072b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.642038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 171/345] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:51.642777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b5770-2b1a-4bdb-bd37-65e3a2b2072b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:51.671487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:51.692218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fdbb8-e3eb-4da3-b51a-ff0645b7e40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:51.692867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fdbb8-e3eb-4da3-b51a-ff0645b7e40c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.693157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fdbb8-e3eb-4da3-b51a-ff0645b7e40c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.693439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fdbb8-e3eb-4da3-b51a-ff0645b7e40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.793614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b5770-2b1a-4bdb-bd37-65e3a2b2072b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:51.810645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fdbb8-e3eb-4da3-b51a-ff0645b7e40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:51.813386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fdbb8-e3eb-4da3-b51a-ff0645b7e40c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:51.834763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:51.835366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.835683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.835963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.849288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:51.856195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:51.857189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:51.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:51.863140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 172/346] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:51.864125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:51.965877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:51.968507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:51.975819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 175/347] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:51.977314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:51.977727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:51.978081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:51.997687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:51.999526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7d651-8978-465c-820e-0fce08dd3ce5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:51.999720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:52.053500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:52.054334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.054719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.055158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:52.102673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 176/348] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:52.103827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:52.163384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:52.165479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:52.290639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d409e31-109e-45ef-a171-bb808f2d3b6f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:52.303746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:52.349933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:52.355602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 173/349] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:52.355602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:52.368545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:52.369812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.369812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.370131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:52.370815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:52.371123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:52.376927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:52.392084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 177/350] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:52.394445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:52.580787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eb31e-c5fd-46b3-8c9e-01687f8d0dc9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:52.609359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744978-ce2e-4721-9810-11beb6270a93 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:52.636503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:52.637195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.637473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.637739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:52.638998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:52.641019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:52.643264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 174/351] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:52 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:52.644386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:52.665599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:52.665599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.665804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.666170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:52.684019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:52.693133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:52.694056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:52.694843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:52.698134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 178/352] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:52.699517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:52.729479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:52.783249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:52.819323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af90e1-0ae7-4867-9c81-1a5c52639ad1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:52.885492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad91195-35fd-4e85-b14b-3ebdbdd23efc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:52.900470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8889daa-3315-4528-994c-e0550a569371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:52.901857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8889daa-3315-4528-994c-e0550a569371 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.902190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8889daa-3315-4528-994c-e0550a569371 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.902676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8889daa-3315-4528-994c-e0550a569371 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:52.905667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8889daa-3315-4528-994c-e0550a569371 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:52.909383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8889daa-3315-4528-994c-e0550a569371 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:52.911886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 175/353] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:52 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:52.913451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3778714-f7c1-4368-813b-97cfb30fd6e0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.913737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3778714-f7c1-4368-813b-97cfb30fd6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.914013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3778714-f7c1-4368-813b-97cfb30fd6e0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:52.915615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 176/354] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:52.916994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3778714-f7c1-4368-813b-97cfb30fd6e0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:52.940446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cadb0d-0a4a-4b73-a29e-724ffdc2e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:52.942030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cadb0d-0a4a-4b73-a29e-724ffdc2e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:52.942388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cadb0d-0a4a-4b73-a29e-724ffdc2e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:52.942666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cadb0d-0a4a-4b73-a29e-724ffdc2e8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.049376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cadb0d-0a4a-4b73-a29e-724ffdc2e8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:53.051241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cadb0d-0a4a-4b73-a29e-724ffdc2e8d5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:53.079885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3778714-f7c1-4368-813b-97cfb30fd6e0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:53.096948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:53.137046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:53.138002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.139010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.139602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.143713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:53.153810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:53.157103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:53.160423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 177/355] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:52 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:53.163394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:53.222515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 179/356] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:53.224064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aaa1f-0e6c-4947-9201-0580bc508793 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.224450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aaa1f-0e6c-4947-9201-0580bc508793 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.224807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aaa1f-0e6c-4947-9201-0580bc508793 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.226480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 180/357] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:53.227165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aaa1f-0e6c-4947-9201-0580bc508793 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:53.348028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfa8b-8f14-4f0d-a8a3-146c1614fbd4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:53.401208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aaa1f-0e6c-4947-9201-0580bc508793 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:53.407335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:53.408878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.408878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.408878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.409426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:53.410203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:53.411146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3fe65-780c-462d-a1be-d9b71b6aae11 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:53.423657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 178/358] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:53.427775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.427775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.427775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.447504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:53.449686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:53.453405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:53.454202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.454490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.454764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.456214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:53.457715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:53.459496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 181/359] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:53 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:53.460359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:53.483875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:53.494975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:53.548725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 179/360] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:53.549483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:53.614895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bff5ee-693f-48ba-b845-37ddfbc7ce60 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:53.683147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:53.684158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.684559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.685018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.686891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:53.687369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:53.689894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54b7c9-6d0e-43f4-bce8-631d6dc112da None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:53.701920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 182/361] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:05:53 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=d47409cb4aa642018317ecedd3a36461 => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:53.704286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.705385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.705385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.722184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0048ee-cb9b-4f33-8b1c-9ba34d999ba3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:53.736341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:53.738259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:53.772801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:53.774097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:53.774613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:53.774956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:53.776580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:53.778332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:53.780985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 180/362] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:53 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:53.782364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:53.850787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:53.890548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 183/363] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:53.892450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:53.911510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:53.955514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df79b-df53-41ab-b7c0-73928c879990 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.019570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.021212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.021497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.021768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.053615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807d9e4-ab91-4e42-8395-ce78da8b1139 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.098351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.099249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.099736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.099973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.101450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=e772eae3dbe343e896c3f799600481b6) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:54.101779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:54.103657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:54.113187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 184/364] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:05:53 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=d47409cb4aa642018317ecedd3a36461 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:54.115600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.147888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:54.150099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:54.202743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:54.225400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:54.258631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad4581-cac8-4b40-8a01-e1ff0ac421ea None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.298461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 181/365] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:54.299448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.320139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.320139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.320139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.320474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.338239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:54.347183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:54.348389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:05:54.349918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:54.355612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 185/366] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:54.362804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.451276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61022e97-03ef-480f-8b97-63013f359a20 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.505011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.506036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.506478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.506922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.508992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:54.511441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:54.515021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 182/367] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:54.515869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.519874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:54.548628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bab639-bc84-421b-a909-ed07a4dee9bb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.603724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.604382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.604662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.604934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.609028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:54.621758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:54.627319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:54.628242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:05:54.629016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:54.634600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 186/368] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:54.635616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.690119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b114d91-d311-4afa-837c-dfe5bedbe060 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.737025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.737662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.737998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.738342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.739583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:54.741027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:54.742723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 183/369] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:54.743522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.817160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d86c42-11bd-4db8-94ba-081a22470839 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.831411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:54.864523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.865545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.866154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.866785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.867973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:54.868518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:54.870131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc81d-c3b4-49bd-963a-e484d34068ea None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:54.871076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:54.887369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 187/370] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:54.889419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.912033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:54.913071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:54.913642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:54.914100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:54.915073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:54.915689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:54.917457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:54.924859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 184/371] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:54.926057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:54.979922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:55.029177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dada-7e5c-43e0-a676-12aff4facddb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:55.088399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:55.089074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:55.089433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:55.089613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:55.090530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:55.090829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:55.092494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:55.110783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf3388-f734-4188-92fe-8ea87e3a5a09 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:55.115596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 188/372] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:55.116610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:55.153203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:55.154293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:55.154762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:55.155222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:55.156864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:55.158725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:55.161200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 185/373] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:54 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:55.162398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:55.172106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:55.300511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f382e-9cee-4929-8f69-21223623cacf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:55.341304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:55.342318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:55.342803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:55.343251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] PATH_INFO: `/v3/users/6c97bf50f080451d8f6f4919e6e059f4` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:55.344781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] RBAC: Authorizing `identity:get_user(user_id=6c97bf50f080451d8f6f4919e6e059f4)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:55.346425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:55.348663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 186/374] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:05:55 2022] GET /identity/v3/users/6c97bf50f080451d8f6f4919e6e059f4 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:55.350087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:55.374508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc1549-f10d-478f-807b-a582206a29e6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:55.432803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:55.444780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:55.474022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d303566-2cc6-46e1-bf4b-40699afb5537 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:55.478122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:55.478122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:55.478385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:55.478630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:55.514535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:55.515603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:55.516178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:55.516651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:55.517639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=e772eae3dbe343e896c3f799600481b6) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:55.518149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:55.519906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:55.527748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 187/375] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:05:55 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:55.528982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:55.640481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:55.642081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:55.753945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8167a-bc15-40d3-8fc8-9fc234e39a3f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:55.833661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:55.834798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:55.835291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:55.835852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:55.837181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] RBAC: Adding query filter params (role.id=4af4e31926f64e829ceca358f3b877f8, user.id=6c97bf50f080451d8f6f4919e6e059f4, scope.system=all) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:55.837770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:55.839947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:55.844074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:55.856564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 188/376] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Aug 1 14:05:55 2022] GET /identity/v3/role_assignments?user.id=6c97bf50f080451d8f6f4919e6e059f4&scope.system=all&role.id=4af4e31926f64e829ceca358f3b877f8 => generated 510 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:55.894699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 189/377] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:05:55.902587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:56.036234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93644b3-23a2-4e74-9d56-148611f66793 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:56.104747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:56.105651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:56.106042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:56.106376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] PATH_INFO: `/v3/projects/045f384688624ccf8339b3c71e2bd890` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:56.108383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] RBAC: Authorizing `identity:get_project(project_id=045f384688624ccf8339b3c71e2bd890)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:56.109858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:56.111992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 189/378] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:55 2022] GET /identity/v3/projects/045f384688624ccf8339b3c71e2bd890 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:56.119077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:56.150711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:56.286188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5b28e-f71a-4339-a835-5ca38a2d0678 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:56.331998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b151-4fed-4cb1-ba4c-e0460d155366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:56.333049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b151-4fed-4cb1-ba4c-e0460d155366 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:56.333651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b151-4fed-4cb1-ba4c-e0460d155366 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:56.334129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b151-4fed-4cb1-ba4c-e0460d155366 None admin] PATH_INFO: `/v3/groups/0809402c26874509ad4bbe49783b85a5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:56.344020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b151-4fed-4cb1-ba4c-e0460d155366 None admin] RBAC: Authorizing `identity:get_group(group_id=0809402c26874509ad4bbe49783b85a5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:56.347117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b151-4fed-4cb1-ba4c-e0460d155366 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:56.351217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 190/379] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:56 2022] GET /identity/v3/groups/0809402c26874509ad4bbe49783b85a5 => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:56.359285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:56.468162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e55b1-8492-4081-9a50-7b884bae79d0 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:56.468810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e55b1-8492-4081-9a50-7b884bae79d0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:56.469287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e55b1-8492-4081-9a50-7b884bae79d0 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:56.472010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 191/380] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:56.479292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:56.480494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:56.480977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:56.510491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:56.521535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:56.522788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:56.563383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51def3-991a-4eff-956b-efe2a7b670c3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:56.649431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:56.650261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:56.655480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:56.655480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:56.655480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] RBAC: Adding query filter params (group.id=0809402c26874509ad4bbe49783b85a5, role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=045f384688624ccf8339b3c71e2bd890) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:56.655480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:56.656502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:56.663506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 192/381] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:56.674854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 190/382] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:05:56 2022] GET /identity/v3/role_assignments?group.id=0809402c26874509ad4bbe49783b85a5&scope.project.id=045f384688624ccf8339b3c71e2bd890&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 260 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:56.690601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:56.717600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:56.858740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf65f85-c228-4adc-84aa-5f0158cd90f0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:56.911664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:56.912687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:56.913150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:56.913603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:56.942421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:56.948339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:56.949322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:56.950097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:56.953542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 193/383] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:56.963704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:57.127586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182828c-8834-4391-ac77-93ec9813e017 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:57.159654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:57.181058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:57.181711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:57.181986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:57.182331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:57.183506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:57.184066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:57.187017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aff524-f105-4819-92e9-e2f36608dec1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:57.208028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 191/384] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:57.214109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:57.310034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d42870-7ece-47c9-ad3a-e6139cf4e3c7 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:57.311060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d42870-7ece-47c9-ad3a-e6139cf4e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:57.311498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d42870-7ece-47c9-ad3a-e6139cf4e3c7 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:57.315943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 192/385] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:57.329409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:57.329817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:57.330079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:57.393640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:57.394750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:57.533936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:57.586864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 193/386] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:57.624067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876a84d-1aff-4e94-8e26-40679be73f60 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:57.627707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:57.751530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:57.752296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:57.754776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:57.754776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:57.757495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:57.758993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:57.766802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 194/387] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:57 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 553 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:57.769912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:57.835814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39719f0-072b-4b10-b609-4fabcadc9164 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:57.857616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:57.892328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:57.893374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:57.893833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:57.895753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:57.919911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:57.925407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:57.927425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:57.929197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 194/388] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:05:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:05:57.936235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:58.006972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef1885-ee2f-4c1a-b951-ebc78b302855 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:58.080205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:58.080880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.081171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.081451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.088307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:58.089784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:58.091947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 195/389] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:57 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:58.100545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:58.127888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:58.154208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa12f5-b564-4500-b0d0-f44037b5387a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:58.243934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:58.244671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.245140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.245529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.246470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:58.246954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:58.249165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:58.262998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 195/390] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:05:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:58.274285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:58.351841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:58.399274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ee057-13fc-4e7a-a765-9d38a655dab6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:58.460411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2d4b4-95f9-4b73-9353-544208585ae5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:58.484628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be43c64-f6b8-4020-9eff-458d8b81ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:58.485312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be43c64-f6b8-4020-9eff-458d8b81ebf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.485589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be43c64-f6b8-4020-9eff-458d8b81ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.485860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be43c64-f6b8-4020-9eff-458d8b81ebf1 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/groups/dbfc97328f8d4484ae272caeca106384/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.487974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be43c64-f6b8-4020-9eff-458d8b81ebf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb545ff39f4342ab40fac4b4f6ad35, group_id=dbfc97328f8d4484ae272caeca106384, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:58.489956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be43c64-f6b8-4020-9eff-458d8b81ebf1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:58.509234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1977d0-e945-436f-8692-fcb787bb6875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:58.509778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 196/391] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Aug 1 14:05:58 2022] PUT /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/groups/dbfc97328f8d4484ae272caeca106384/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:58.511160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c86f9-9ad3-493c-8f40-50d091e09a3f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.511445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c86f9-9ad3-493c-8f40-50d091e09a3f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.511759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c86f9-9ad3-493c-8f40-50d091e09a3f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.513488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 197/392] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:58.516015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1977d0-e945-436f-8692-fcb787bb6875 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.516302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1977d0-e945-436f-8692-fcb787bb6875 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.516635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1977d0-e945-436f-8692-fcb787bb6875 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.518152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1977d0-e945-436f-8692-fcb787bb6875 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:58.519733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1977d0-e945-436f-8692-fcb787bb6875 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:58.522807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 196/393] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:58 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:58.529164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c86f9-9ad3-493c-8f40-50d091e09a3f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:58.532420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.532703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.532979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.560657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:58.563138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:58.738335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 197/394] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:05:58.763952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:58.771021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:58.817626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c86f9-9ad3-493c-8f40-50d091e09a3f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:58.830275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:58.891386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:58.892497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:58.892929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:58.893382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:58.895709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:58.898601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:58.902319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 198/395] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:58 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:58.909626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:58.952940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eef562-cce2-40cd-9ac2-3aa5751d7f32 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:59.058044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81df05-dde4-4480-820d-5544e203af0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:59.059112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81df05-dde4-4480-820d-5544e203af0e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.059637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81df05-dde4-4480-820d-5544e203af0e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.060778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81df05-dde4-4480-820d-5544e203af0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.077911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81df05-dde4-4480-820d-5544e203af0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0d81df05-dde4-4480-820d-5544e203af0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:59.084461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:59.085364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:05:59.086134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 198/396] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:59.091530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c06dde-116a-42dc-84ff-d1d839d4dc94 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.091530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c06dde-116a-42dc-84ff-d1d839d4dc94 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.091530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c06dde-116a-42dc-84ff-d1d839d4dc94 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.091530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 199/397] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:59.094775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c06dde-116a-42dc-84ff-d1d839d4dc94 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:59.133794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad91648-7803-4de8-af7f-4f593eac2ed4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:59.176910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66099c-bae8-4237-be2a-43b5817ace4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:59.177990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66099c-bae8-4237-be2a-43b5817ace4b None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.178475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66099c-bae8-4237-be2a-43b5817ace4b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.178967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66099c-bae8-4237-be2a-43b5817ace4b None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6/groups/dbfc97328f8d4484ae272caeca106384/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.181335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66099c-bae8-4237-be2a-43b5817ace4b None admin] RBAC: Authorizing `identity:create_grant(project_id=e772eae3dbe343e896c3f799600481b6, group_id=dbfc97328f8d4484ae272caeca106384, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:59.183047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66099c-bae8-4237-be2a-43b5817ace4b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:59.202349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:59.225232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 199/398] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Aug 1 14:05:58 2022] PUT /identity/v3/projects/e772eae3dbe343e896c3f799600481b6/groups/dbfc97328f8d4484ae272caeca106384/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:05:59.227413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.227973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.228497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.243336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:59.255074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c06dde-116a-42dc-84ff-d1d839d4dc94 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:59.272639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:59.274616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:59.345052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:59.347562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.348160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.349977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.351874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:59.352449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:59.354443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:59.365934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 200/399] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:59.372293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:59.444387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 200/400] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:05:59.502072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:59.537972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268e28c-5852-48f1-86b5-2b5841bfabe5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:59.596778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:59.597347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.597631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.597902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.601306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:59.602983 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:59.605037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 201/401] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:05:59 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:59.613439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:59.673716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:59.730342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10be7a-e088-4d16-ae28-cb36017e124f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:59.825294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:59.853916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:59.854993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.855494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.856020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.884060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0f10-ebe4-4871-ab31-cf5d199d8484 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:59.894795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:05:59.905539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:05:59.907291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:05:59.908881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:05:59.913515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 201/402] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:05:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:05:59.923787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:59.970288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:59.971198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:59.971793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:59.972284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:59.976729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:59.979400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:59.983309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 202/403] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:05:59 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:05:59.991917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:00.163557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3434a9e-368c-4509-81b8-03a0a89e432b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:00.207293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:00.211567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:00.212857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:00.213444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:00.214163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:00.215309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:00.215760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:00.217783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a8d32-2961-4923-a7c6-f8437c22f654 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:00.222647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd822e-adce-4da1-bbb9-0273a767cf1c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:00.231513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 202/404] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:05:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:00.231513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e953cc-81e4-499f-a1d0-92fde41ccfa7 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:00.231513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e953cc-81e4-499f-a1d0-92fde41ccfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:00.231513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e953cc-81e4-499f-a1d0-92fde41ccfa7 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:00.231513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 203/405] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:00.237077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:00.237422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:00.237773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:00.259126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:00.260799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:00.299603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:00.302497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:00.303583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:00.304209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/groups/dbfc97328f8d4484ae272caeca106384/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:00.308411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9fb545ff39f4342ab40fac4b4f6ad35, group_id=dbfc97328f8d4484ae272caeca106384, role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:00.312781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:00.327865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:00.344753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 203/406] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Aug 1 14:05:59 2022] PUT /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35/groups/dbfc97328f8d4484ae272caeca106384/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:00.346436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:00.371490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 204/407] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:06:00.422859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:00.586814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a495c6-2298-4bef-a2ca-33253f6e71c6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:00.587019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bf2ae-7d3b-415d-9832-ba9974bb42ba None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:00.641951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:00.642622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:00.642906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:00.643175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:00.644546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:00.646029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:00.647764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 204/408] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:00 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:00.656585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:00.674802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:00.675390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:00.675698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:00.676037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:00.709857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:00.719841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:00.721660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:00.723231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:00.726461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 205/409] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:00.727511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:00.729440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:00.819253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:00.938051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c7ad5-9fef-4b6b-8c89-00fe5e955284 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:00.976689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad898d-461c-4c57-8dd8-7e602004b867 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:01.011495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:01.012165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:01.012449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:01.012721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:01.014457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:01.017083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:01.020318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 205/410] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:00 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:01.028588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:01.056507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:01.057556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:01.058048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:01.058538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:01.063496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:01.063496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:01.063843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:01.081539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 206/411] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:01.092190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:01.246241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:01.286647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fd15e-01f8-4c0b-949d-add8b44b43ba None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:01.290899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:01.302809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784ba5c-1af2-4272-9174-9322dd5af159 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:01.361020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:01.362738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:01.363541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:01.364298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] PATH_INFO: `/v3/groups/0809402c26874509ad4bbe49783b85a5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:01.367174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] RBAC: Authorizing `identity:get_group(group_id=0809402c26874509ad4bbe49783b85a5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:01.369958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:01.380036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 207/412] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:01 2022] GET /identity/v3/groups/0809402c26874509ad4bbe49783b85a5 => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:01.383027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:01.438802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:01.440842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:01.441739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:01.445829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6/groups/dbfc97328f8d4484ae272caeca106384/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:01.449361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] RBAC: Authorizing `identity:create_grant(project_id=e772eae3dbe343e896c3f799600481b6, group_id=dbfc97328f8d4484ae272caeca106384, role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:01.453423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:01.487337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 206/413] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Aug 1 14:06:01 2022] PUT /identity/v3/projects/e772eae3dbe343e896c3f799600481b6/groups/dbfc97328f8d4484ae272caeca106384/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:01.624924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b3c7e-bcbe-44db-a3e0-bc58959b744b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:01.684608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efff81f-d19a-4e60-8e5b-0e353aa166e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:01.685486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff81f-d19a-4e60-8e5b-0e353aa166e0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:01.685859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff81f-d19a-4e60-8e5b-0e353aa166e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:01.686230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff81f-d19a-4e60-8e5b-0e353aa166e0 None admin] PATH_INFO: `/v3/projects/045f384688624ccf8339b3c71e2bd890` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:01.687924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efff81f-d19a-4e60-8e5b-0e353aa166e0 None admin] RBAC: Authorizing `identity:get_project(project_id=045f384688624ccf8339b3c71e2bd890)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:01.689743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efff81f-d19a-4e60-8e5b-0e353aa166e0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:01.699691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 208/414] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:01 2022] GET /identity/v3/projects/045f384688624ccf8339b3c71e2bd890 => generated 344 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:01.701902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:01.895760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507ac1f-5d1b-490f-b45e-8444e611cd79 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:01.953882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229eaf3-2fd3-44d5-8aa9-ef112b1313da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:01.954703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229eaf3-2fd3-44d5-8aa9-ef112b1313da None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:01.955085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229eaf3-2fd3-44d5-8aa9-ef112b1313da None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:01.955590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229eaf3-2fd3-44d5-8aa9-ef112b1313da None admin] PATH_INFO: `/v3/projects/045f384688624ccf8339b3c71e2bd890/groups/0809402c26874509ad4bbe49783b85a5/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:01.958191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229eaf3-2fd3-44d5-8aa9-ef112b1313da None admin] RBAC: Authorizing `identity:create_grant(project_id=045f384688624ccf8339b3c71e2bd890, group_id=0809402c26874509ad4bbe49783b85a5, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:01.959716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229eaf3-2fd3-44d5-8aa9-ef112b1313da None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:01.984592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 207/415] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Aug 1 14:06:01 2022] PUT /identity/v3/projects/045f384688624ccf8339b3c71e2bd890/groups/0809402c26874509ad4bbe49783b85a5/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:06:02.147800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153510c-62bf-43ab-9b08-7e1c274d222d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:02.149199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153510c-62bf-43ab-9b08-7e1c274d222d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:02.149969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153510c-62bf-43ab-9b08-7e1c274d222d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:02.153301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 209/416] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:02.161533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:02.161908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:02.162096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:02.191254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:02.192928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:02.205249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:02.206364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320e0d9-4de7-42f8-913f-d50aff3424e5 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:02.206664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320e0d9-4de7-42f8-913f-d50aff3424e5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:02.206929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320e0d9-4de7-42f8-913f-d50aff3424e5 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:02.209364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 210/417] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:02.218152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:02.218918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:02.219358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:02.259768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:02.262754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:02.330953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 208/418] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:02.346432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:02.357786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:02.409099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 211/419] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:06:02.440589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:02.537100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e010a86-fa13-480b-bd28-0a17cfa24924 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:02.585742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:02.591036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:02.591036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:02.591036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:02.629439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25d9a6-e079-468e-8cf6-9ad55d418934 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:02.640244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:02.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:02.663157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 209/420] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:02.677269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:02.696588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:02.705711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:02.706572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:02.706862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:02.707139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:02.752766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:02.762157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:02.764879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:06:02.766942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:02.768950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 212/421] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:02.778633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:02.864654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:02.976032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6e6aa-2dec-40ba-bdcc-bae5e21e1886 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:03.050649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:03.051839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:03.052291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:03.052732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:03.053978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:03.054477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:03.055865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774cdb2-6422-4ddb-bf3b-3201cf03e8e6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:03.057178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:03.074671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 210/422] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:06:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:03.083923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:03.173876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:03.174697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:03.174881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:03.175263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:03.175876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:03.176189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:03.177681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:03.202850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 213/423] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:06:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:03.207565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:03.209609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:03.257082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03f9ee-f8f0-4a58-876c-9976eeddbec1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:03.327150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:03.328251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:03.328743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:03.329974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:03.452881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:03.455266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:03.555092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:03.600950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 211/424] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:06:03.609798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:03.633415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:03.689819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a223-c15a-4a5e-af8b-42e8405b2967 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:03.804522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:03.805527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:03.805822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:03.806214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:03.809239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c292-03ca-4414-b32c-868031c7dd33 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:03.873099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc8d3d-9edc-478f-b8f1-6451497865a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:03.873973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc8d3d-9edc-478f-b8f1-6451497865a4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:03.874353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc8d3d-9edc-478f-b8f1-6451497865a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:03.874731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc8d3d-9edc-478f-b8f1-6451497865a4 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:03.876309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc8d3d-9edc-478f-b8f1-6451497865a4 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:03.877932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc8d3d-9edc-478f-b8f1-6451497865a4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:03.880416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 212/425] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:03 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:03.885951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179517f-eebb-44dc-a094-91d06ddbcd0c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:03.885951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179517f-eebb-44dc-a094-91d06ddbcd0c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:03.885951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179517f-eebb-44dc-a094-91d06ddbcd0c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:03.885951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 213/426] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:03.888252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179517f-eebb-44dc-a094-91d06ddbcd0c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:03.952263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:03.953101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:03.955511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:04.077984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179517f-eebb-44dc-a094-91d06ddbcd0c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:04.134484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f562f92-f51c-4f4b-806d-9768495644f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:04.135861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f562f92-f51c-4f4b-806d-9768495644f4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.136302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f562f92-f51c-4f4b-806d-9768495644f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.136724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f562f92-f51c-4f4b-806d-9768495644f4 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.138809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:04.139853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f562f92-f51c-4f4b-806d-9768495644f4 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:04.143309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f562f92-f51c-4f4b-806d-9768495644f4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:04.146706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 214/427] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:03 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:04.148360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.148782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.149215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.171689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 214/428] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 960 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:06:04.171689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:04.186017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:04.187697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:04.321145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 215/429] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:04.322980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:04.373827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee019f-091f-41cf-82f2-fc336b790ce0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:04.384903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:04.444125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:04.444793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.444907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.445245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.446093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=e772eae3dbe343e896c3f799600481b6) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:04.446334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:04.455520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:04.471136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 215/430] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:06:04 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=d47409cb4aa642018317ecedd3a36461 => generated 611 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:04.472416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:04.565325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:04.607105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02cfc8-ab79-41fa-b77f-b6dac5e147d7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:04.646038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176fa63-bc34-496e-97ce-c9807a876740 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:04.712581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f75826-32c2-498b-a061-eea56044124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:04.713934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75826-32c2-498b-a061-eea56044124b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.714598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75826-32c2-498b-a061-eea56044124b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.715333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f75826-32c2-498b-a061-eea56044124b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.723128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abed956-4ca8-4a87-acd4-90b0a38c688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:04.724775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed956-4ca8-4a87-acd4-90b0a38c688b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.725370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed956-4ca8-4a87-acd4-90b0a38c688b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.725962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed956-4ca8-4a87-acd4-90b0a38c688b None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.727900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed956-4ca8-4a87-acd4-90b0a38c688b None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:04.729808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed956-4ca8-4a87-acd4-90b0a38c688b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:04.738734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 216/431] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:04 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:04.738734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ab0e2-c6cd-42eb-875d-3266381b61f1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.738734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ab0e2-c6cd-42eb-875d-3266381b61f1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.738734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ab0e2-c6cd-42eb-875d-3266381b61f1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.738734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 217/432] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:04.744427 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ab0e2-c6cd-42eb-875d-3266381b61f1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:04.748590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f75826-32c2-498b-a061-eea56044124b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-74f75826-32c2-498b-a061-eea56044124b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:04.756741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:04.758339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 216/433] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:06:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:04.760330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.761521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.761521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.797952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:04.799823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:04.909220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:04.915858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ab0e2-c6cd-42eb-875d-3266381b61f1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:04.922397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 217/434] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:04.924989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:04.976680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:04.977888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:04.978394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:04.979015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:04.980627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:04.983023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:04.984833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 218/435] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:04 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:04.985648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:05.011956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:05.086395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb14973-c658-4e9a-b65e-3a27dd554b31 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.151958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:05.152692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.152976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.153262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.154516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:05.154881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:05.156366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:05.167232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 218/436] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:05.167850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:05.168185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b75c4-91f7-4492-b9fe-615e6cf172da None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.226055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f6bd0-ea54-4c2b-be91-2641923da310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:05.227868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f6bd0-ea54-4c2b-be91-2641923da310 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.228214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f6bd0-ea54-4c2b-be91-2641923da310 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.228488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f6bd0-ea54-4c2b-be91-2641923da310 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.242248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f6bd0-ea54-4c2b-be91-2641923da310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b66f6bd0-ea54-4c2b-be91-2641923da310 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:05.247496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:05.248936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 219/437] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:06:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:05.250230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66e5c6-e54e-4d30-b570-834f61666783 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.251127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66e5c6-e54e-4d30-b570-834f61666783 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.251127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66e5c6-e54e-4d30-b570-834f61666783 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.252414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 220/438] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:05.253038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66e5c6-e54e-4d30-b570-834f61666783 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:05.299685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:05.334310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241c8c8-e973-4ccf-b087-20013852ec53 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.344656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:05.381801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:05.382933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.383396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.383902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.385326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=d47409cb4aa642018317ecedd3a36461, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:05.385854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:05.388705 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:05.399105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 219/439] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:06:05 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=d47409cb4aa642018317ecedd3a36461 => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:05.400594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:05.430707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66e5c6-e54e-4d30-b570-834f61666783 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.487728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:05.488376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.488656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.488932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.592776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc4ec-5406-42c9-9a3b-015c1a56ffe4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.603749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:05.605906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:05.651679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:05.662534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:05.663345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.664444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.664444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.664798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:05.665111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:05.666755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49e579-cf9d-4cfb-8f93-d76bda979a4c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:05.677825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 220/440] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:05.679432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.679432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.679835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:05.711583 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:05.713463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:05.751413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:05.775320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 221/441] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:05.776242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:05.804301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 221/442] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:05.805178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:05.919811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec8af-2e2a-4ef6-8682-3ae308647672 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.954916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373df8d-79ba-4fec-9a19-759659003c66 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:05.958037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:05.981332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:05.982101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:05.982549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:05.982994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.012168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.013466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.013808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.014125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] PATH_INFO: `/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.015674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] RBAC: Authorizing `identity:get_project(project_id=c9fb545ff39f4342ab40fac4b4f6ad35)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.017593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.019408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 222/443] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:05 2022] GET /identity/v3/projects/c9fb545ff39f4342ab40fac4b4f6ad35 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:06.020882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.080339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:06.106677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.109370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.185631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eafdac-9424-413b-b082-e03f4c1ca38c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.238110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.238700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.238985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.239266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.252460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:06.258015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:06.259719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:06.261455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 223/444] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:06.262737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.282257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 222/445] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:06.283635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.328458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:06.386663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:06.399223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8dee-0a91-4f93-88b6-9abee724ea60 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.425902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3612c16-a0cc-45dc-a5a9-9bc09b58d1df None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.441026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.441717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.441995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.442270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.443834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.445320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.447071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 224/446] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:06.447941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.474867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.475203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.475502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.475780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.476513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:06.476815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.478240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.488525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 223/447] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:06.489213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.572601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c47b-a693-4bc1-9e47-431a08e56369 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.593960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:06.610305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e1f14-56ca-410a-9ea0-59cb6960b953 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.623376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.623376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.623376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.623376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] PATH_INFO: `/v3/projects/e772eae3dbe343e896c3f799600481b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.623894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] RBAC: Authorizing `identity:get_project(project_id=e772eae3dbe343e896c3f799600481b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.624018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.626892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 225/448] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/projects/e772eae3dbe343e896c3f799600481b6 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:06.628325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.668624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.669786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.670066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.670337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.671127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:06.671440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.672929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.678070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:06.679519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 224/449] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=c9fb545ff39f4342ab40fac4b4f6ad35&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:06.691646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:06.828738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c853f7-1ab5-43da-92e1-13363ac475e0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.895286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.895968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.896248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.896526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.900842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73827e80-fd8d-496b-b329-8d9fc4d112d5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:06.973589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:06.975240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:06.975929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:06.976432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] PATH_INFO: `/v3/groups/dbfc97328f8d4484ae272caeca106384` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:06.978524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] RBAC: Authorizing `identity:get_group(group_id=dbfc97328f8d4484ae272caeca106384)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:06.981048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:06.983204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 225/450] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/groups/dbfc97328f8d4484ae272caeca106384 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:06.990442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:07.006984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:07.008679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:07.020711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:07.046467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:07.143680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851bf5-2945-4558-8604-198dc046fe0a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:07.165663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 226/451] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:07.177570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:07.194547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:07.195584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:07.195937 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:07.196418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:07.197461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] RBAC: Adding query filter params (group.id=dbfc97328f8d4484ae272caeca106384, role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=e772eae3dbe343e896c3f799600481b6) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:07.197907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:07.200389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:07.212917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 226/452] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:06:06 2022] GET /identity/v3/role_assignments?group.id=dbfc97328f8d4484ae272caeca106384&scope.project.id=e772eae3dbe343e896c3f799600481b6&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:07.322851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c5752-88c4-4c0d-9b8a-8d19a618e3c6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:07.355637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:07.367479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:07.368278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:07.368628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:07.368970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] PATH_INFO: `/v3/projects/045f384688624ccf8339b3c71e2bd890` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:07.370377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] RBAC: Authorizing `identity:get_project(project_id=045f384688624ccf8339b3c71e2bd890)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:07.372104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:07.374207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 227/453] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:06:07 2022] GET /identity/v3/projects/045f384688624ccf8339b3c71e2bd890 => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:07.384076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:07.492236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:07.514579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893289b-2f24-42b5-bfd6-c7aa39867868 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:07.562782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12683184-485d-43d2-8bb3-05f843d2db87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:07.563815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683184-485d-43d2-8bb3-05f843d2db87 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:07.564256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683184-485d-43d2-8bb3-05f843d2db87 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:07.564691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683184-485d-43d2-8bb3-05f843d2db87 None admin] PATH_INFO: `/v3/groups/0809402c26874509ad4bbe49783b85a5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:07.566569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683184-485d-43d2-8bb3-05f843d2db87 None admin] RBAC: Authorizing `identity:get_group(group_id=0809402c26874509ad4bbe49783b85a5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:07.568976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683184-485d-43d2-8bb3-05f843d2db87 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:07.571972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 227/454] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Aug 1 14:06:07 2022] GET /identity/v3/groups/0809402c26874509ad4bbe49783b85a5 => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:07.581058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:07.737250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f2f47-3a5a-4ed4-9162-234eab156921 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:07.781014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:07.781677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:07.782066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:07.782344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:07.783215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] RBAC: Adding query filter params (group.id=0809402c26874509ad4bbe49783b85a5, role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=045f384688624ccf8339b3c71e2bd890) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:07.783724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:07.785372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bf7a2-21af-4219-b504-d49052d13f06 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:07.792802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 228/455] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Aug 1 14:06:07 2022] GET /identity/v3/role_assignments?group.id=0809402c26874509ad4bbe49783b85a5&scope.project.id=045f384688624ccf8339b3c71e2bd890&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:10.634637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37263a15-d3e7-4cbb-96a4-b046128c0a26 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:10.635335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37263a15-d3e7-4cbb-96a4-b046128c0a26 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:10.635335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37263a15-d3e7-4cbb-96a4-b046128c0a26 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:10.636845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 228/456] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:10.645329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28a121-410a-426b-9a01-76988240b1ae None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:10.646036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28a121-410a-426b-9a01-76988240b1ae None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:10.646036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28a121-410a-426b-9a01-76988240b1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:10.676546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bd28a121-410a-426b-9a01-76988240b1ae None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:10.677789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bd28a121-410a-426b-9a01-76988240b1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:10.790485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:10.811660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 229/457] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:10.883795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37263a15-d3e7-4cbb-96a4-b046128c0a26 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.032204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37263a15-d3e7-4cbb-96a4-b046128c0a26 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:11.044916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545880ef-ca94-4af2-9e57-82d0cedca5d8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.045931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545880ef-ca94-4af2-9e57-82d0cedca5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.046606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545880ef-ca94-4af2-9e57-82d0cedca5d8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.049485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 230/458] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:11.057568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee83605-e16c-453f-9355-c5e990682327 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.058378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee83605-e16c-453f-9355-c5e990682327 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.059049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee83605-e16c-453f-9355-c5e990682327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.087584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1ee83605-e16c-453f-9355-c5e990682327 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:11.090036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1ee83605-e16c-453f-9355-c5e990682327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:11.091263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:11.091974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.092245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.092523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.107088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:11.108924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e88ae6b7-d3c0-4828-b56d-d5499d1394b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:11.161693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:11.163836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:11.166593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:11.166593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:11.166593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:11.166593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:11.166593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 229/459] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:11.167132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d178c9a-50db-46bc-a85d-68bbb8c587f9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.167265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d178c9a-50db-46bc-a85d-68bbb8c587f9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.169729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d178c9a-50db-46bc-a85d-68bbb8c587f9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.172737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 230/460] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:11.174629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d178c9a-50db-46bc-a85d-68bbb8c587f9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.181844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:11.234345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 231/461] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:11.236943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.237408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.237820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.274760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:11.276090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:11.342702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d178c9a-50db-46bc-a85d-68bbb8c587f9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:11.371488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 232/462] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:11.371488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.414248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:11.415239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.415817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.416277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.417202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:11.417715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:11.421687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:11.443227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 231/463] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:11.444501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.464071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:11.500103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b67ac3-b343-4fe0-910e-61095eb4680d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:11.541694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:11.542693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:11.544455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.545181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.545655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.559183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:11.561589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:11.579024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:11.579980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:11.580874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:11.580874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:11.580874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:11.580874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:11.580874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 233/464] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:11.581137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.610752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa52cb7-55ab-41a2-aa83-4b4efe63fa6c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:11.668388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:11.671654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.672101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.672658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.689594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:11.691492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-86844aca-6cff-4a96-b402-abaf29192007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:11.711987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:11.712923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 232/465] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a085a-bf39-4b00-bfa3-2aa05e2e6d6a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a085a-bf39-4b00-bfa3-2aa05e2e6d6a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a085a-bf39-4b00-bfa3-2aa05e2e6d6a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.724045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 233/466] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:11.725038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a085a-bf39-4b00-bfa3-2aa05e2e6d6a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.743190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:11.747286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31fe3-0f8e-46cf-8547-e0bcca07a250 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:11.788742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:11.789435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.789724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.790078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.790701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:11.792448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:11.831493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 234/467] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:06:11 2022] POST /identity/v3/users => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:11.831493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:11.862750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a085a-bf39-4b00-bfa3-2aa05e2e6d6a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:11.892113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:11.920620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:11.921927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.922389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.922844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.924604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:11.925124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:11.927346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08daa5-ec50-456b-88af-c000ee82a139 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:11.954372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 234/468] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:11.956198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:11.956625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:11.956974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:11.978975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:11.980447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:11.995263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ebcb7-2fb0-4d8e-b291-4ebe54d5f237 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:12.053022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:12.054055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:12.055166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:12.055671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:12.057002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:12.057509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:12.059306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:12.073519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 235/469] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:12.074652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:12.084905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 235/470] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:12.092675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:12.121307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:12.253447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc79b-7bb3-4750-a0b8-df99ed2e98b3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:12.257777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543ce79-e6bd-4c74-96ac-c42e4889b1b9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:12.266242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:12.302450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:12.303248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:12.303617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:12.303939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:12.304716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:12.306278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:12.309421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167480db-732f-4bf7-95ee-ac1048f37c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:12.310888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167480db-732f-4bf7-95ee-ac1048f37c8d None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:12.311365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167480db-732f-4bf7-95ee-ac1048f37c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:12.311851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167480db-732f-4bf7-95ee-ac1048f37c8d None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:12.312720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167480db-732f-4bf7-95ee-ac1048f37c8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:12.314634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167480db-732f-4bf7-95ee-ac1048f37c8d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:12.334916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 236/471] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:06:12 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:12.341333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:12.343950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 236/472] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:06:12 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:12.509765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:12.579684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cfab5-ce11-4a74-b08d-b4f321e84dbd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:12.626497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:12.627685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:12.628185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:12.628655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:12.648490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:12.650772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:12.678709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:12.680700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:12.682312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:12.682312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:12.682312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:12.682312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:12.686433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 237/473] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:12.692749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fed34c-895c-41d3-aabc-a28e0033b7b0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:12.693755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fed34c-895c-41d3-aabc-a28e0033b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:12.694261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:12.694493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fed34c-895c-41d3-aabc-a28e0033b7b0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:12.696552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 237/474] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:12.703002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:12.703347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:12.703770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:12.730945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:12.733616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:12.880662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 238/475] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:06:12.918098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b1271-1de5-4409-9f5e-8520cd06a99f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:12.926973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:13.003526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:13.023135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:13.023801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:13.024085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:13.024370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:13.025086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:13.025395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:13.026793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:13.051309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 238/476] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:13.060477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:13.219245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7468400-5624-4a1a-8803-f228d42bec64 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:13.228012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:13.258269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aca94-8b19-4c28-b2ab-884db60530ba None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:13.273437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:13.279430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:13.280305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:13.281087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:13.282652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:13.288307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:13.354627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 239/477] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:06:13 2022] POST /identity/v3/users => generated 249 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:13.365198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:13.365852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:13.366141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:13.366410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:13.367155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:13.368674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:13.383201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.sql.core [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 14:06:13.383201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 14:06:13.383201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [parameters: {'id': 'f5def4af4546478b933bbac238ef1d71', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 14:06:13.383201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80524) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 01 14:06:13.407486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:06:13.409220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application yield resource Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise self.value Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application session.commit() Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 14:06:13.411063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.flush() Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application compat.raise_( Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 14:06:13.414825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:06:13.423880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:06:13.427023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [parameters: {'id': 'f5def4af4546478b933bbac238ef1d71', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:13.430133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 14:06:13.441168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:13.444329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 239/478] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Aug 1 14:06:12 2022] POST /identity/v3/roles => generated 144 bytes in 491 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:13.444545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:13.563420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:13.688787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf780e4-3335-41d9-8dfc-0b1e01dc30fa None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:13.752465 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d41f91-0023-4beb-a0e1-22dc13db81c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:13.753136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d41f91-0023-4beb-a0e1-22dc13db81c8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:13.753411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d41f91-0023-4beb-a0e1-22dc13db81c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:13.753688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d41f91-0023-4beb-a0e1-22dc13db81c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:13.770167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d41f91-0023-4beb-a0e1-22dc13db81c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-18d41f91-0023-4beb-a0e1-22dc13db81c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:13.775760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:13.777452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:13.780214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 240/479] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:06:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:13.790457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:14.001517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee738c-f7c6-4540-a919-cddc6abdf56f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:14.073777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:14.075864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:14.076652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:14.077403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:14.079100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:14.079960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:14.082795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbade865-c361-4238-8338-78c2f6d6e22b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:14.105806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 240/480] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:15.190222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11222a6-3826-48f2-8fac-3b0eaba1aaaa None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.190979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11222a6-3826-48f2-8fac-3b0eaba1aaaa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.190979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11222a6-3826-48f2-8fac-3b0eaba1aaaa None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.192540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 241/481] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:15.198435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.199174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.199174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.219641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:15.221018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:15.239350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:15.311618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 241/482] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:06:15.332934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11222a6-3826-48f2-8fac-3b0eaba1aaaa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:15.479946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:15.487861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11222a6-3826-48f2-8fac-3b0eaba1aaaa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:15.532130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:15.532781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.533065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.533341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.547822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:15.553554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:15.555207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:15.556882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:15.561927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 242/483] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:15.562198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:15.670706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f441758-569a-47c5-8412-9d6515e23030 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:15.708189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:15.708820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.709131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.709372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.710097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:15.710402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:15.711824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72205ea-485f-41ce-93e9-8ef1cc86a7d4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:15.725690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 242/484] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:15.727112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:15.881647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb627-08b4-42f1-93af-995cdafa225b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:15.936222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:15.936883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.937168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.937448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.949264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11ce2-ec07-474d-b94d-b1c2e2e1cf85 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.949921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11ce2-ec07-474d-b94d-b1c2e2e1cf85 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.950405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11ce2-ec07-474d-b94d-b1c2e2e1cf85 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.952385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 243/485] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:15.955491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:15.955886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:15.960592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8aaf3-0658-479d-a5d6-2a01f92bff7f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.961175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8aaf3-0658-479d-a5d6-2a01f92bff7f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.961717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8aaf3-0658-479d-a5d6-2a01f92bff7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-90ea7092-3283-4ce6-adbf-767d0c736ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:15.975214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:15.976246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:15.977051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:15.977051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:15.977051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:15.977051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:15.977051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 243/486] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:15.983680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748f23e-59cc-43c5-bc0d-f710ea639060 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:15.983967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748f23e-59cc-43c5-bc0d-f710ea639060 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:15.984259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748f23e-59cc-43c5-bc0d-f710ea639060 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:15.986256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 244/487] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:15.987423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748f23e-59cc-43c5-bc0d-f710ea639060 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:15.989253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c6f8aaf3-0658-479d-a5d6-2a01f92bff7f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:15.991145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6f8aaf3-0658-479d-a5d6-2a01f92bff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:15.994810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.006401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.103529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 244/488] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:06:16.105645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.106137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.106642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.132630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:16.134394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:16.147249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748f23e-59cc-43c5-bc0d-f710ea639060 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.209051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.210563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.211102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.211648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.213157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:16.213919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:16.215866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:16.235639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 245/489] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:16.237446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.243757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 245/490] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:16.250204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.347925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.349070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.383729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0eaf2-29b5-4eeb-8767-3da2c5c035eb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.394836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de94250-b63a-4e6d-8817-6b968db2a9be None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.441636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.442417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.442774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.443126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.445496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.446340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.446731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.447193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.451489 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:16.453142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:16.459724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:16.465041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:16.465914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:16.466961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 246/491] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:16.467684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:16.469223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:16.471471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 246/492] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:16.475673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.608055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771145e-d9dc-4472-9de8-d77efe760898 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.626002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43805195-51df-4591-95d8-b5808cdb0426 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.647401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.656870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.658095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.658526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.658944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.676207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:16.678741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:16.685911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:16.687114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:16.687942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:16.688693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.688693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.688693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.688693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.690178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:16.690602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:16.691268 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 247/493] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:16.692371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.698854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:16.714703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 247/494] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:16.718301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.851328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f87d0-4d3e-421a-92c0-8f3156d8645c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.857243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0c577-09c5-411a-9a87-07777c7a286b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:16.900447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.901262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.901709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.902051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.902880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:16.903986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:16.904879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:16.905647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:16.905913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:16.906364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:16.906807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:16.909006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:16.909528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:16.912055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:16.922050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 248/495] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:16.923310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:16.979561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 248/496] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:16.979561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.006984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.039008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.080248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53e3a9-0f40-4cfd-b35e-ea7b18a7e911 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.102944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e94868-6d69-4995-ae11-71dd6da5defa None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.131484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.133152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.133653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.134392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.145438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.146067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.146418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.146767 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.150907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.152647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.158934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.160544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:17.178010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:17.179557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:17.181513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:17.181513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:17.181513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:17.181513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:17.183159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:17.185070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:17.186994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:17.186994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:17.186994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:17.186994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:17.187384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 249/497] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:17.187384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.192237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 249/498] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:17.194006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.331921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.335443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81d64b-4106-4b26-a896-ea15c706f87f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.348204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6bee0-63b1-4014-9ac0-46c827f70b83 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.362358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.382888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d63c2-086d-4bbd-89d0-8f0b2000176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.383563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d63c2-086d-4bbd-89d0-8f0b2000176a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.383889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d63c2-086d-4bbd-89d0-8f0b2000176a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.384118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d63c2-086d-4bbd-89d0-8f0b2000176a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.410287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.411730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.412783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.412783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.414377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:17.414936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.417616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.435588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 250/499] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:17.440635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.510439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d63c2-086d-4bbd-89d0-8f0b2000176a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.513337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d63c2-086d-4bbd-89d0-8f0b2000176a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.579775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb70584-56e6-47f5-af31-7c99e19cde28 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.625078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.625705 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.625987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.626309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.626973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:17.627279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.628732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dafaf-b37f-4d81-803c-2bdef3c9c20f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.640966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 251/500] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:17.642668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596286c7-b3be-427d-b36e-7ed9bbae79c5 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.643133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596286c7-b3be-427d-b36e-7ed9bbae79c5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.643611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596286c7-b3be-427d-b36e-7ed9bbae79c5 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.645109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 252/501] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:17.646310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596286c7-b3be-427d-b36e-7ed9bbae79c5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.669222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.675761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 250/502] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:17.676976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016a86f-fcec-4f3c-aa03-186ddb036355 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.677262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016a86f-fcec-4f3c-aa03-186ddb036355 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.677533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016a86f-fcec-4f3c-aa03-186ddb036355 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.678863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 251/503] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:17.679592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016a86f-fcec-4f3c-aa03-186ddb036355 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.688435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.768506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596286c7-b3be-427d-b36e-7ed9bbae79c5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.807003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.807670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.807957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.808239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.817554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.817939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016a86f-fcec-4f3c-aa03-186ddb036355 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:17.819043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-713236d9-6af9-45ed-976a-eb56772ac853 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:17.831773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:17.833004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 253/504] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:17.834252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9c79-f601-4aca-b86c-2e027f2b46e4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.835674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9c79-f601-4aca-b86c-2e027f2b46e4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.835674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9c79-f601-4aca-b86c-2e027f2b46e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.856035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-cdcb9c79-f601-4aca-b86c-2e027f2b46e4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:17.857915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cdcb9c79-f601-4aca-b86c-2e027f2b46e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:17.868180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:17.868839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.869114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.869388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.877291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:17.878754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:17.893920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:17.894843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:17.895955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 252/505] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:17.896490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:17.948607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 254/506] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:17.950657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:17.951128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:17.951599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:17.978412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:17.979857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:17.985972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:17.997845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.033281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a66ea9-c088-4bf2-9ce4-75a9c92f1dfc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.061943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 255/507] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:18.063192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.074112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.074773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.075100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.075380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.090005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.092151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:18.110653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:18.112231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:18.114000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:18.114000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:18.114000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:18.114000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:18.115685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 253/508] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:18.118249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.218970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395319-3d5f-466b-836d-c88ee36bd371 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.256627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.257247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.257596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.257849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.258725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:18.259039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.260564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:18.262416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cef4-26ac-4981-9e65-004b93998dfc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.283674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.301479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 256/509] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:18.302367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.310201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.310830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.311111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.311394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.312392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:18.312902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.315655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:18.316757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.337967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 254/510] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:18.338828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.426727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d690c-4fee-48b4-aeae-83892be5683a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.449237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688da54-1180-4125-8626-4a30bb2ce21c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.462940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.463631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.463910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.464183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.475435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:18.480550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:18.482159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:18.482976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 257/511] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:18.483651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.490187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.490851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.491136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.491414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.503016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.504524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:18.521184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:18.522119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:18.522990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:18.522990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:18.522990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:18.522990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:18.522990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 255/512] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:18.528748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.554100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.566262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.602473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a111f91-5f86-4487-a297-0480d40b511e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.636982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39324f-8b86-4864-812c-3e5fb47aa0e6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.639536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.640258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.640514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.640797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.641507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:18.641877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.643405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:18.659799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 258/513] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:18.661017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.685057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.685772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.686050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.686320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.784136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.786778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:18.794659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61da61-d3df-4b02-be15-8f7cc41c88c5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:18.819739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.828158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:18.845868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:18.846936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:18.847352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:18.847868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:18.924533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 256/514] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:18.925394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:18.944378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:18.945861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.069488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 259/515] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:19.069488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.072399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f1f5-b1c1-49b7-abcf-0c031c1fee40 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.107747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:19.114986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:19.133708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.134624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.135057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.135379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.136298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:19.136658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.139033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.148946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 257/516] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:19.149923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.219033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957e4b6-c8dc-4a26-8320-be722e1ba450 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.269726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963681-123d-4b32-a6d6-968a23d3afdf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.270559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.271217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.271591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.271878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.272595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:19.272903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.274315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.288121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 260/517] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:19.288958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.324359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.325586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.325880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.326253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.333976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.336464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:19.350586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:19.352903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 258/518] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:19.354997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.414282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:19.415409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:19.438412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2cd19-9fd9-4cf5-ba63-3604f69a6707 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.495241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.497749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.497749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.497749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.516854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.519361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.525399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be1343-8698-42bc-8b41-c3107e303e69 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:19.539991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:19.540924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:19.541739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:19.541739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:19.541739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:19.541739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:19.541739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 261/519] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:19.542002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.570033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.570680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.570967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.571244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.586666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.588166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:19.606503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:19.608160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:19.609793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:19.609793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:19.609793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:19.609793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:19.609793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 259/520] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:19.610706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.676189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d4e0-af53-4b30-bbe5-40ccfa8b96ce None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.717356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.718005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.718288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.718566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.721015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:19.726653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:19.728344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19add74c-2ea0-42ff-857d-0243e3c71750 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.731634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.732807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:19.753236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:19.754411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:19.755243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:19.755243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:19.755243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:19.755243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:19.755243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 262/521] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:19.755560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.770924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.771648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.771917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.772272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.772843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.774289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.802929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 260/522] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:06:19 2022] POST /identity/v3/users => generated 247 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:19.803886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:19.900339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76060-4d8a-469d-81c8-babbb244187e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.957444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9be5a6-2f62-4266-bf81-ef7c329d293b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:19.959338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:19.960868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:19.960868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:19.960868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:19.962157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:19.962479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:19.964818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:19.997191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 263/523] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:19.999483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.011166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.011996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.012306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.013322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.014060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:20.014377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.016423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.031327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 261/524] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:20.032831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.063796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:20.100082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:20.187269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e731f02-cc34-493b-b13a-4fc0f083b56a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.210191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc3c10-86b9-4b6d-a313-0f407c2f56e3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.258115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.259105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.259601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.260051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.261228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:20.261735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.264695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.283043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.284684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.284684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.284684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.285399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:20.285805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.287285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.296857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 264/525] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:20.298442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.310931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 262/526] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:20.310931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.407776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:20.463803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c82c-ec3a-4995-bca8-6ae30176e299 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.483321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e806b1-81f0-49fd-8e6e-af19922d54a2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.502659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:20.528408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.529510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.529961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.530237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.539977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce272dd-250f-48af-8242-896099350d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.540959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce272dd-250f-48af-8242-896099350d32 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.541429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce272dd-250f-48af-8242-896099350d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.541880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce272dd-250f-48af-8242-896099350d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.543341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce272dd-250f-48af-8242-896099350d32 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=15b5f371cd614e3bb246b015818a97f2) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:20.544703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce272dd-250f-48af-8242-896099350d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.547609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce272dd-250f-48af-8242-896099350d32 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.552203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.554543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.568152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 265/527] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/role_assignments?user.id=15b5f371cd614e3bb246b015818a97f2&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:20.568152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce272dd-250f-48af-8242-896099350d32 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:20.588639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:20.589661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:20.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 263/528] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:20.595506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.765805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce272dd-250f-48af-8242-896099350d32 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.779999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ae5f6-62fb-4ef7-ab06-362511f1a4fa None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:20.810030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.811323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.812019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.813165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.828978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.832593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.838483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:20.850140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:20.851692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:20.852381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:20.853048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:20.858912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:20.860929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 266/529] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:20.863200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.875850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:20.878760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:20.903402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:20.905386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:06:20.907416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:20.911571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 264/530] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:20.913528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:20.934788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:21.050116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084a7d-e8d1-49ea-9236-a9d6cc8c4da9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:21.057397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b622-c1d9-414c-9dd0-6dca3a159a24 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:21.101010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:21.101943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:21.102386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:21.102821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:21.104002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:21.104578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:21.106387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:21.107070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:21.108031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:21.108431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:21.108818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:21.109967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:21.110404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:21.113148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:21.130629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 265/531] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:21.132202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:21.147817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 267/532] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:21.149446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:21.206712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:21.287717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c6e7b-138d-4778-996e-606bd0c182c1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:21.304451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:21.318824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee303f-a9da-49c2-a385-326caa7e201c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:21.354156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:21.355431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:21.355838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:21.356359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:21.357556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:21.357962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:21.360585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:21.369539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:21.370426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:21.371072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:21.371528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:21.374199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=ee96f24e85de4092b31081d44fe0a03d) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:21.374910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:21.376727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:21.384375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 266/533] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Aug 1 14:06:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:21.385423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 268/534] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:21 2022] GET /identity/v3/role_assignments?user.id=ee96f24e85de4092b31081d44fe0a03d&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:21.387602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:21.392465 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:21.565258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c788eda-4812-443a-8b32-04640702f334 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:21.581726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:21.620756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:21.621395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:21.621764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:21.622556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:21.623381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=f3a7a9a0bda04d128a4f045fdb0bc80c) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:21.624999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:21.625212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:21.632896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 267/535] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:21 2022] GET /identity/v3/role_assignments?user.id=f3a7a9a0bda04d128a4f045fdb0bc80c&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:21.671568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e21d5-8a50-4d53-a903-7c93172cd837 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:21.715880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:21.716927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:21.717431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:21.717918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:21.752379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:21.807332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:21.809276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:21.990749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 269/536] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:21.997431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:22.205998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6028134-7d21-4c78-8684-a465015edca6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:22.253672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:22.254333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:22.254620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:22.254898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:22.271392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:22.272901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-03331ff7-50f7-4ff6-87f3-461fc36d7fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:22.292008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:22.293009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:22.293853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:22.293853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:22.293853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:22.293853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:22.293853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 268/537] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:22.298975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:22.416304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:22.498637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f2186-2280-4514-baa8-8e294a536370 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:22.565697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:22.566323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:22.566953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:22.566953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:22.568171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:22.568502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:22.569151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fe986-4751-40cb-82d1-07425c421214 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:22.569877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fe986-4751-40cb-82d1-07425c421214 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:22.570152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fe986-4751-40cb-82d1-07425c421214 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:22.570999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:22.578587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 269/538] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:22.587128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:22.587619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:22.588825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:22.593521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 270/539] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:22.615871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:22.625476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:22.627583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:22.718390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:22.883211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8fdc-d29a-4c83-b577-8580c029ab2a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:22.914410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 270/540] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:22.939923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:22.953443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:22.982905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:22.983885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:22.984327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:22.984754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.002985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:23.004858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:23.023168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:23.024206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:23.025163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:23.027396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 271/541] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:23.033645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:23.204383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186656f0-bc2e-44cc-b84b-6f762bca6aaa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:23.276557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fe34b-6170-4f14-86a3-57eeb38e4a26 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:23.288865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:23.289698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.290095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.290476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.315828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:23.329283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:23.331051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:23.332548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 271/542] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:23.337753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:23.351683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:23.353330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.354210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.354951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.356938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:23.357792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:23.360928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:23.385629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 272/543] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:23.393542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:23.398382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:23.555851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:23.666166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183a76e-03b6-409b-a6e5-d6781bcdc6c6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:23.692878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08e4b-315e-4f03-aef3-1fca890b0dd2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:23.725999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:23.727342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.728030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.728621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.729613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:23.730137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:23.732174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5efe-bbae-4058-b4d3-354064d724ab None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:23.745937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:23.747032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.747574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.747679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 272/544] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:23.748291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.749372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=771f49a069014fdc956e073c2d0a0fdd) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:23.749925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbddd3-bb76-459b-85a7-4ed4b3725226 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.750266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:23.750448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbddd3-bb76-459b-85a7-4ed4b3725226 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.750911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbddd3-bb76-459b-85a7-4ed4b3725226 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.752936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253c0c-9b3c-49bd-8cc9-06cae7be3c27 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:23.753176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 273/545] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:23.757399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbddd3-bb76-459b-85a7-4ed4b3725226 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:23.766094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 273/546] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:23 2022] GET /identity/v3/role_assignments?user.id=771f49a069014fdc956e073c2d0a0fdd&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:23.767932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.768497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.769016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.810773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:23.812493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:23.864087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:23.898260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbddd3-bb76-459b-85a7-4ed4b3725226 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:23.944951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:23.945900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:23.947092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:23.947628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:23.962209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:23.964189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:23.982845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:23.983786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:23.984689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:23.984689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:23.984689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:23.984689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:23.991571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 274/547] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:23.993811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.013686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 274/548] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:24.050361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.120155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:24.138523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4e167-2682-4932-83ec-314e198fed9c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:24.180864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:24.189571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:24.190580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.191065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.191568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.192905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:24.193461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:24.195979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:24.212161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 275/549] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:24.219244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.343697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b1fd1-524e-4aeb-a97b-443504f056cb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:24.347697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79700122-a3b7-438b-a666-be51d74f4557 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:24.387317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:24.388353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.388835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.389463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.399234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:24.401217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:24.415828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:24.416959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:24.417830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:24.422536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 276/550] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:24.429019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.482953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:24.484087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.484575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.485044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.491789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:24.532970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:24.550307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd385e6-82b8-4671-a562-fc8900ff3439 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:24.551386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:24.552463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:24.553257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:24.559604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 275/551] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:24.562844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.597545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:24.599557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.600076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.600553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.601564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:24.602075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:24.603755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:24.627541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 277/552] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Aug 1 14:06:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:24.634551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.780840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:24.781118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a79f-6107-400b-b6a4-0c656922ee6c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:24.795818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:24.804645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d5cb2-98da-4048-b30d-adc164adf3e6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:24.835559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50716-15dd-4f70-b939-25e92295e10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:24.836199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50716-15dd-4f70-b939-25e92295e10a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.836483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50716-15dd-4f70-b939-25e92295e10a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.836945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50716-15dd-4f70-b939-25e92295e10a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.875763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:24.877182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.877964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.878775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.880553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:24.881298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:24.884412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0103-68be-458d-af08-1da78e89d54d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:24.907536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 276/553] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:24.907536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de306ab7-ac79-44d2-ad2b-dae7ab704ab7 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:24.907536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de306ab7-ac79-44d2-ad2b-dae7ab704ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:24.907536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de306ab7-ac79-44d2-ad2b-dae7ab704ab7 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:24.911622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 277/554] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:24.912750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de306ab7-ac79-44d2-ad2b-dae7ab704ab7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:24.945935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50716-15dd-4f70-b939-25e92295e10a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:24.947631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50716-15dd-4f70-b939-25e92295e10a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:25.044340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de306ab7-ac79-44d2-ad2b-dae7ab704ab7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.089312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.089967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.090801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.091080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.097470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 278/555] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:06:25.097470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.097470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.097470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.108486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:25.109974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:25.126282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b9f73af5-0e74-4add-8b82-9d8ac66ae052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:25.127740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:25.128801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:25.129652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:25.129652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:25.129652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:25.129652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:25.129993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:25.131648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 278/556] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:25.132094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:25.133814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c307bb-b1b9-4bc1-a3d5-53d3c92b08c9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.134311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c307bb-b1b9-4bc1-a3d5-53d3c92b08c9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.135055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c307bb-b1b9-4bc1-a3d5-53d3c92b08c9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.137144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 279/557] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:25.141564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c307bb-b1b9-4bc1-a3d5-53d3c92b08c9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.158573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:25.241686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 279/558] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:25.243413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.301412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c307bb-b1b9-4bc1-a3d5-53d3c92b08c9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.347158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.348365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.348968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.349412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.363108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:25.364627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-56b5fc59-d053-42ac-8a62-19809d8cf4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:25.381860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:25.384785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 280/559] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.388773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.421144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:25.423195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:25.426637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da3bf-a0bb-47c6-a1ea-b4bc914a337f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.484225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.484871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.485210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.485426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.486115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:25.486419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:25.487904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:25.495633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:25.504388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 280/560] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:25.505705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.513718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:25.539497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 281/561] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:25.539497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.669653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20531e6-2960-482a-b644-b22045903f27 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.682489 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96ca0b-15d9-490c-b6a1-120066ca3868 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.709434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.710258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.710573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.710808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.721569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.722321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.722673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.723025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.723188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:25.724358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:25.724673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:25.726216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:25.729863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:25.731655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:25.732550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 281/562] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:25.733931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.746835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 282/563] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:25.748311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.808436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:25.811020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:25.875320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c4361-e873-476d-b0f0-6697d2303afc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.875621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc895a-f0ef-4e7f-9e95-5c733c296255 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:25.926030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.926744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.927030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.927305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.929352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:25.930673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:25.931126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:25.931610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:25.936392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:25.938360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:25.951700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:25.952006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:25.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 282/564] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:25.962697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:25.963865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:25.965894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 283/565] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:25.967358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.080799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dc59e-cd34-4ac3-8fd5-8f77b71dd5d8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.081629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d77ed-ba0f-4ef3-92b5-273abb364aa4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.091666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.103026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.122017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.122662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.123008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.123239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.124193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.124829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.125109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.125395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.126130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:26.126443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.127880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.131155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.132699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.139342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 283/566] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:26.141125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:26.146833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:26.147788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:26.148757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 284/567] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:26.149256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.272021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c620529-1437-4901-a896-d682bcf6cd04 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.290987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a50c-2458-453e-82f3-475f0149ba41 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.326306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.327158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.327447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.327840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.328764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:26.329079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.330502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.333143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.333828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.334119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.334395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.335200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:26.335528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.337064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.349203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 285/568] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:26.350192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.361991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 284/569] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:26.361991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.385473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.433022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.479906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db3db9-fae8-4a2d-b19e-dc516bd1aff9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.525005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.525582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3351b1-e0f5-46c5-aaab-2e87a5da0254 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.525700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.525801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.526070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.537626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.539212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:26.556513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:26.557497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:26.558473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:26.558473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:26.558473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:26.558473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:26.558473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 286/570] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:26.560418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.565028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.565669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.565940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.566210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.567217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:26.567706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.569306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.583618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 285/571] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:26.584392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.677947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.708764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414670ec-f5b5-4099-8d82-0f39c17a03a2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.709934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.714097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d262c84-76c6-4274-902f-87093fc2e547 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.758946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.759995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.760403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.760852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.767711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.768707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.769151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.769595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.779755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.781266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.783877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.785377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-443623c0-182a-451b-87c6-16e6c35f29af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:26.801019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:26.802201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:26.803883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:26.803883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:26.803883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:26.803883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:26.804300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:26.805800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:26.808768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:26.808768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:26.808768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:26.808768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:26.811962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 287/572] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:26.812123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 286/573] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:26.812123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.812359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f109f-203b-4589-904f-67a0070b771a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.812359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f109f-203b-4589-904f-67a0070b771a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.812359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f109f-203b-4589-904f-67a0070b771a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.812359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 288/574] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:26.812359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f109f-203b-4589-904f-67a0070b771a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.926718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510b6ad-c23c-4bc5-8498-7e0e0697dc88 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.933327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f109f-203b-4589-904f-67a0070b771a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:26.963388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.964040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.964327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.964635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.965402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:26.965713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.967104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.969748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:26.970452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:26.970742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:26.971025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:26.971865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=3690b0c864cb48a4b55774bb8875b644) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:26.972186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:26.973932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:26.976038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:26.985823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 287/575] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:26.985823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:26.991719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 289/576] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/role_assignments?user.id=3690b0c864cb48a4b55774bb8875b644&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:26.993063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.000661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.133251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f22a0-ede9-42bf-b496-85208d2219ea None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.150071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a60c4-0cb9-4e74-b5ad-1a99855db371 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.181437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.181986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.182269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.182541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.183241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:27.183589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.185492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94b41a-ee09-4911-91ac-7348b638db0d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.198839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 288/577] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:27.200153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.200368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.200648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.210200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.211186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.211736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.212228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.213124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:27.213628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.215758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.222497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:27.224046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:27.230548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 290/578] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:27.232517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.302443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.305129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.317687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 289/579] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:27.317687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.374654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87095db-9517-4295-aba7-1156fd93f21a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.420556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.421361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.421642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.421936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.430053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.431630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.434685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a94d8-5774-4c98-86a8-a2e8c5c7b1ba None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:27.445218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:27.446274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:27.447205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 291/580] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:27.447828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.486281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.486939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.487224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.487528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.495306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.496976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:27.509534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:27.511162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 290/581] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:27.512874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.584821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc7782-5d8b-4176-979a-3a9cb93b1d6c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.596396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.617521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.639777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.640629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.640913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.641186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.650294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.651812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.660699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe774-affe-45ea-a02d-03c4f834b9c9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:27.668155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:27.669868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:27.672069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 292/582] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:27.674452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.716669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.717246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.717589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.717876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.737313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:27.743048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:27.744322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:27.745151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 291/583] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:27.745824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.821385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f7f32-3faf-4b16-8fae-fd78141abe8d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.861553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.862172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.862449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.862719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.863748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:27.864137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.865893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.871309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7851e9-3223-418c-9037-71be84003516 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:27.886467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 293/584] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:27.887936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:27.900261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.917710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:27.918542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:27.919009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:27.919534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:27.920696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:27.921116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:27.922684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:27.928212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:27.941237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 292/585] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:27.941237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.009630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ae22-e70f-40b4-a931-2527d6e7a0e5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.048658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe90b-3ec9-4c31-8bf4-910b41a0c490 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.050164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.050957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.051242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.051555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.052471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:28.052808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.054414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.075565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 294/586] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:28.075760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.094579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.095357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.095665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.095958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.096650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:28.096949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.098936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.112370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 293/587] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:28.112370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.158180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:28.190780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:28.203420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337135-7ef2-4bba-89df-363ed71232e5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.226694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da956-29c1-4f0d-9f8a-778a668aa1ac None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.241974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.242893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.243336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.243634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.256117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.257585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.267010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.267733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.268014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.268305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/15b5f371cd614e3bb246b015818a97f2/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:28.277131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:28.278146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:28.279143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:28.279143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:28.279143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:28.279143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:28.279143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 295/588] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:28.279874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.290334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=15b5f371cd614e3bb246b015818a97f2, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.292503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.319493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 294/589] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:28 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/15b5f371cd614e3bb246b015818a97f2/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:28.319493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.420964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e818a-c198-4df2-b26c-59e8d151862c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.455932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:28.464114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12135309-9e7e-497d-ae70-656a9636af5f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.472928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.473923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.474207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.474483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.493034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.494936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.504864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:28.517185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:28.518682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:28.520382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:28.520382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:28.520382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:28.520382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:28.520382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 296/590] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:28.520382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.530814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.531852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.536003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.536519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.554206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.556137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:28.579860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:28.581090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:28.582474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:28.582474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:28.582474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:28.582474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:28.585910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 295/591] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:28.594509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.714645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34478-978c-43c6-b7cf-5940cacb333d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.740029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dca3c-cb52-42ee-9596-48e508b9c27c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.758271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.759267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.759726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.760268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.761725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:28.762364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.765496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.783515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 297/592] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:28.784388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.794704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:28.794704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:28.795010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:28.795083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:28.799479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:28.799479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:28.799479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:28.816165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 296/593] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:28.817409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:28.821784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:28.874182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:28.953677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00bc30-68fc-4359-b4ac-64eb2318a428 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:28.987987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd3c32-8a44-47bf-bbd3-839a1446e00f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.010314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.011247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.011579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.011855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.013106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:29.013577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.015318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.035868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 298/594] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:29.037357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.041347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.042495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.042934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.043421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.052585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.054453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:29.071202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:29.072349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:29.073316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:29.075853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 297/595] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:29.076761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.173021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:29.188534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ede3d-6afb-4afe-9404-f027ab50857d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.211602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:29.232418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445d5f3-283a-47d0-9052-101fd2f45a67 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.239203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.240049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.240432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.240802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.253996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.256894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.275185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.276147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.276147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.276437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.286431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:29.286787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:29.288431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:29.289748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:29.290175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.290255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 299/596] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:29.290770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0d437b0f-7284-4cb3-a011-9aedd10b5e00 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:29.307975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:29.309022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:29.310109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 298/597] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:29.311989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59988c04-9487-4117-b2a7-182155bbc14e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.311989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59988c04-9487-4117-b2a7-182155bbc14e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.311989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59988c04-9487-4117-b2a7-182155bbc14e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.312601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 299/598] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:29.313335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59988c04-9487-4117-b2a7-182155bbc14e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.440862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c780-4d6c-47ad-8bd4-aad34982fa18 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.455808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59988c04-9487-4117-b2a7-182155bbc14e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.484439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:29.489496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.490129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.490580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.490862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.492001 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:29.492313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.494024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.497802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.498516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.498785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.499056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.499863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:29.500230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.501890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81871441-0daf-479f-8cfe-da085d76bd60 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.518429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 300/599] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:29.518429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.527177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 300/600] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:29.528631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.528978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.529316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.549906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:29.551068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:29.578911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:29.639960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 301/601] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:29.641409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.682109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685238d-3079-4189-93a9-308f6ee8c12a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.732971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.733606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.733896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.734166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.734983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:29.735335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.736953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.759068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 301/602] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Aug 1 14:06:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:29.759996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.812158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce01f-62d8-4a9c-8b73-bc1644fb161b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.830436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:29.856688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.857373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.857664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.857970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/ee96f24e85de4092b31081d44fe0a03d/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:29.885787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=ee96f24e85de4092b31081d44fe0a03d, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:29.889135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:29.903774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:29.912331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0558f-f6bc-49f2-b4be-b3acd937370d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:29.923726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 302/603] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:29 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/ee96f24e85de4092b31081d44fe0a03d/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:29.923726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:29.988856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:29.990187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:29.990669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:29.991136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/f3a7a9a0bda04d128a4f045fdb0bc80c/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:30.019250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=f3a7a9a0bda04d128a4f045fdb0bc80c, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:30.021974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:30.041656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 302/604] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:29 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/f3a7a9a0bda04d128a4f045fdb0bc80c/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:30.043060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:30.101255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40997c-08c0-4bf9-8830-32987ce98c6f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:30.184845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:30.186333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:30.186997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:30.187686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:30.214159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:30.228067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:30.229933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:30.237085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 303/605] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:30.242674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:30.295232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:30.306132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2994690-9404-4de3-ad53-9a8664415934 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:30.360491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:30.361039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:30.361315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:30.361629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:30.373891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:30.375262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:30.383201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:30.406899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:30.408665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:30.410445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:30.410445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:30.410445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:30.410445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:30.410445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 303/606] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:30.416591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:30.459593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ad49-b89b-4e6e-8dec-bae6444dd7f0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:30.521770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:30.523267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:30.524024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:30.524769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:30.526215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:30.527024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:30.530201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:30.547441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 304/607] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:30.555759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:30.630996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522b247-3570-4461-ab1d-7061fca949b9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:30.682153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:30.683167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:30.683781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:30.684811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:30.685719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:30.686238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:30.687860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:30.697033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:30.702907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 304/608] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:30.717455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:30.723753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c809-9fd0-4919-b67e-a786987ecf4f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:30.766644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:30.767311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:30.767624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:30.767901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:30.800964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:30.803674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:30.832481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:30.833514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:30.834605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:30.834605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:30.834605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:30.834605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:30.834605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 305/609] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:30.841249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:30.872138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:30.970800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598d185-1308-4364-b32a-65762d2d8753 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:31.049433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca101854-27da-4471-a5e3-bfa6923d2a07 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:31.085358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:31.086292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.086733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.087189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.104914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:31.106374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:31.107388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.107795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.108173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.108998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:31.109583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:31.115496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:31.115780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:31.128126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:31.139556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:31.140961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:31.142396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:31.145939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 305/610] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:31.150021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 306/611] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:31.152437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:31.158496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:31.297574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08146-ab8b-4712-bc8d-9b4f1ef64381 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:31.319884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:31.331710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e16f1-d3db-4718-a65c-5bc5087dc624 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:31.342984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:31.343985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.344488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.344999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.345993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:31.346539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:31.348839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c724f-26ae-44fb-9b9d-40a50a5b0c50 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:31.367813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 306/612] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:31.369752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce051fc-3fa6-4f29-8dae-bc2a2ac9107a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.370274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce051fc-3fa6-4f29-8dae-bc2a2ac9107a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.370760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce051fc-3fa6-4f29-8dae-bc2a2ac9107a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.377596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 307/613] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:31.381724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce051fc-3fa6-4f29-8dae-bc2a2ac9107a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:31.389189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:31.390329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.390828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.391414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.403858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:31.407388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-fac154d5-d3f7-41c5-9446-b9b528fec99e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:31.424751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:31.425740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:31.426703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:31.430616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 307/614] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:31.432333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.432874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.433369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.461202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:31.462831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:31.521242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce051fc-3fa6-4f29-8dae-bc2a2ac9107a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:31.549070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:31.579562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 308/615] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:31.580933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:31.588403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:31.591296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.591613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.591905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/771f49a069014fdc956e073c2d0a0fdd/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.628678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=771f49a069014fdc956e073c2d0a0fdd, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:31.632205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:31.675765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 308/616] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:31 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/771f49a069014fdc956e073c2d0a0fdd/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:06:31.677880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:31.745089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6066b0-beef-481a-b389-a966ad659b0b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:31.760602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:31.785125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:31.785764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:31.786047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:31.786325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:31.787130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:31.787433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:31.788929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:31.835000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 309/617] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Aug 1 14:06:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:31.842489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:31.932274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:31.948849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dd96c-2d89-41ef-bc4c-e4c0d8cce004 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.000881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cbb90-7767-4dc6-8d30-486cab08c09e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.011706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.012888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.013515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.013928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.030210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:32.035827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:32.036790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:32.037627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 309/618] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:32.046196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.057182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.057897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.058182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.058450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.073606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.075340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:32.108976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:32.110476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:32.111383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:32.111383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:32.111383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:32.111383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:32.111383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 310/619] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:32.126565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.225380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:32.250654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350b609-a598-4ba6-9d11-c52bb0b1060a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.272998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:32.280291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f45d7-0515-42f9-badc-3b6a54ce7d87 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.321766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.322553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.322898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.323312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.324272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:32.324858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.326499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.346517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 311/620] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:32.348697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.349419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.349730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.349882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.350726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:32.351121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.352388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.352592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.374575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 310/621] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:32.382671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.496054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b67bc-4736-4bb9-ab58-a8bc709bd686 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.544764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.546390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.546911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.547401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.555781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.557733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:32.571105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:32.572158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:32.573161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:32.575517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 312/622] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:32.584934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.599398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb603427-5123-4690-8ca8-31d7645df9a7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.637250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:32.644872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.645545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.645830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.646111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.680520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.683376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:32.721103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:32.722237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:32.723373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:32.723373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:32.723373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:32.723373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:32.725369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 311/623] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:32.735524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:32.737671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:32.836382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a84983-99bc-4479-9ebb-860802a81701 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.882228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.883034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.883444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.883892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.885081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:32.885499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.888797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c3088-7550-45f9-a190-f959f7fef5dd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.901678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 313/624] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:32.903977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595079-1da1-4d33-a2da-d72975084209 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.904373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595079-1da1-4d33-a2da-d72975084209 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.904755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595079-1da1-4d33-a2da-d72975084209 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.906421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 314/625] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:32.907768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b7743-588a-4777-9779-73a75097ea7c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:32.912827 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76852bad-26bd-4fd4-accc-9404d8c4bcb7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.913193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76852bad-26bd-4fd4-accc-9404d8c4bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.913400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76852bad-26bd-4fd4-accc-9404d8c4bcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.933365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-76852bad-26bd-4fd4-accc-9404d8c4bcb7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:32.935018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-76852bad-26bd-4fd4-accc-9404d8c4bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:32.953463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:32.954104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:32.954392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:32.954669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:32.955383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:32.955729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:32.957221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:32.980328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 312/626] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:32.980328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.015388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 315/627] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:33.016923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c94c-f795-4f9d-82e1-a2e2d4c7dd94 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.017363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c94c-f795-4f9d-82e1-a2e2d4c7dd94 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.017823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c94c-f795-4f9d-82e1-a2e2d4c7dd94 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.019509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 316/628] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:33.020417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c94c-f795-4f9d-82e1-a2e2d4c7dd94 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.034066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.061672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.106185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ed878-07a3-41f3-b95b-582ee538fdf1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.143491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c94c-f795-4f9d-82e1-a2e2d4c7dd94 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.148343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2d239-0d82-42e6-8f7f-6f699fe21f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.149024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2d239-0d82-42e6-8f7f-6f699fe21f36 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.149310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2d239-0d82-42e6-8f7f-6f699fe21f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.149587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2d239-0d82-42e6-8f7f-6f699fe21f36 None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/3690b0c864cb48a4b55774bb8875b644/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.166849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2d239-0d82-42e6-8f7f-6f699fe21f36 None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=3690b0c864cb48a4b55774bb8875b644, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:33.168364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2d239-0d82-42e6-8f7f-6f699fe21f36 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:33.183265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 313/629] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:32 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/3690b0c864cb48a4b55774bb8875b644/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:33.184282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.184990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.185275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.185605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.186385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.186749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.187103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.202152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:33.204333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:33.210759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:33.212516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:33.221113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:33.222371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:33.223319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:33.227772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 317/630] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:33.228785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.334958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.347964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 314/631] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:33.349629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.385015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed80f50-91be-44ae-b972-5c36f0033ff9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.430282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.435770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.436493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.436725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.436997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.457307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:33.463222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:33.465129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 318/632] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:33.466816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.524464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3548159-ad38-4074-b8f8-c106b7f6bb05 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.570634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.571268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.571581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.571861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.572728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:33.573101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:33.575265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:33.600283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 315/633] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:33.601198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.637673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2b2de-d931-40a4-baf2-87526ed33a43 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.678538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.690130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.690813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.691171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.691803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.707872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:33.715895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:33.717115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:33.717962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:33.718585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 319/634] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:33.719250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.733857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.737588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aba9b-ed0c-43bd-a89a-4e7fdfdaaa33 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.783028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.783690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.784005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.784277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.784984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:33.785292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:33.787261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:33.803559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 316/635] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:33.803559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:33.878974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f490f-be86-4fab-b379-a480cb8ffd27 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.915693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.916355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.916642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.916923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.939603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d543f-58d5-4356-a742-c5bbe01b9e8a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:33.981703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:33.982249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:33.982534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:33.982813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:33.983575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:33.983889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:33.985574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:33.996188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:33.999685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 317/636] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:34.000799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.014577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.016310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.039084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:34.145073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1fc6-a0cc-4af7-b809-1c258a0d865d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.160652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 320/637] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:34.160652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.189782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.190948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.191421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.191924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.208126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.210380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:34.234560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:34.235552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:34.236439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:34.236439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:34.236439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:34.236439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:34.236439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 318/638] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:34.237189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.271660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:34.290282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86989d-04a4-4098-a9ca-fe51925f74aa None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.339017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:34.339580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.342653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.342948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.343809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.357740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.359159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:34.376619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:34.377580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:34.378472 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:34.378472 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:34.378472 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:34.378472 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:34.382698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 321/639] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:34.383779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.384242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5906-4436-443f-b7ed-77e9aef805fa None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.437327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.438924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.439585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.440097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.457585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.459065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:34.477573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:34.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:34.479524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:34.479524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:34.479524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:34.479524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:34.483812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 319/640] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:34.484909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.525392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338710d-8e02-4b9d-a27e-15e4799e5acc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.564841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.565626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.566032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.566463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.567201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:34.567653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.569154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.580741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:34.588668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 322/641] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:34.589686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.644327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf588ac-387b-4747-89ce-a45b3d4b592d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.683301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:34.697386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.698543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.699229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.700297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.701184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:34.701570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.703154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ee23d-d36e-47c2-909a-a24e6f7c8580 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.724503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 320/642] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:34.724503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5476-9e27-458f-9efe-7f1df6ac6990 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.724503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5476-9e27-458f-9efe-7f1df6ac6990 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.724503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5476-9e27-458f-9efe-7f1df6ac6990 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.724503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 321/643] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:34.724503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5476-9e27-458f-9efe-7f1df6ac6990 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.759947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465c7e5-71c4-41ac-b7f4-34e917d7f011 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.802064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.803119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.803666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.804155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.805432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:34.805974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.809147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.827775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 323/644] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:34.828125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:34.893191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5476-9e27-458f-9efe-7f1df6ac6990 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.915402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:34.937360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:34.938271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.938772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.939193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.948732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:34.951728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b7822cd6-c645-4df6-be02-93b0506a7395 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:34.966514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:34.968472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:06:34.970697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:34.972613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 322/645] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:34.974398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:34.974687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:34.974970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:34.978891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917254ca-e0d9-4098-b9f6-0f4937a188ea None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:34.996868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:34.998336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:35.035420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.036035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.036324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.036608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.046932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.048680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:35.064598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:35.065645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:06:35.067335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:35.070959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 324/646] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:35.072251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.090965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:35.149004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 323/647] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:35.149870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.248910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729977d2-5093-4a0a-b7ab-696e0acd8a51 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.281023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:35.309020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.309959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.310390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.310743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.318523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c2132-bba4-4987-abfd-d5b42bff1ecb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.319894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.321389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:35.340127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:35.341929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:35.343628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:35.344552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 325/648] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:35.345794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.392413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.393250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.393647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.394043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.395131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:35.395574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.397225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.424657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 324/649] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:35.424657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.467108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:35.487907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6100f-0e6e-409a-95e9-2fa546140765 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.540675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.541460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.541834 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.542484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.543360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:35.543738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2860f4a-d37f-4088-89ac-0cbf7147fde6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.543928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.545564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.566744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 326/650] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:35.567515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.591929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.592588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.592872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.593152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.605389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:35.610425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:35.611690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 325/651] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:35.613124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.618988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:35.701249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f778e-d2e7-47af-92f4-2226aa294302 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.732951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:35.740849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef50a83-ecb7-4cad-8d3a-38285eb3b318 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.742447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.743074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.743366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.743666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.744429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:35.744735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.746888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651f51f-4c4c-4da9-ba34-591729bbb566 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.759634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 327/652] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:35.760869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd390-771e-4d47-ae57-d699b93fdbb0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.761239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd390-771e-4d47-ae57-d699b93fdbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.761618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd390-771e-4d47-ae57-d699b93fdbb0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.762896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 328/653] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:35.763668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fd390-771e-4d47-ae57-d699b93fdbb0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:35.786218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.786837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.787121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.787398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:35.874519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:35.875948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:35.894305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:35.899755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fd390-771e-4d47-ae57-d699b93fdbb0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:35.943346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c1a50-b2ab-4599-8179-f0b6157c3a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:35.944052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c1a50-b2ab-4599-8179-f0b6157c3a47 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:35.944296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c1a50-b2ab-4599-8179-f0b6157c3a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:35.944564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c1a50-b2ab-4599-8179-f0b6157c3a47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.012015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.012213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 326/654] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:36.013469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.030969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c1a50-b2ab-4599-8179-f0b6157c3a47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.032844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c1a50-b2ab-4599-8179-f0b6157c3a47 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.145368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceaeca-6315-4cc5-936c-919563c23e93 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.157088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.162751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 329/655] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:36.164098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.164607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.165069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.184250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:36.185477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:36.185619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.186355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.186539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.186949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.187588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:36.187894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.189282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.200931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 327/656] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:36.202356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.262565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 330/657] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:36.263423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.280773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.330139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6515fc-5000-4bdf-85a3-1955ede9afbc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.367010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.367664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.367951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.368232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.369029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=15b5f371cd614e3bb246b015818a97f2) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:36.369342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.370791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.370970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88a8d5-0271-4252-8150-9a35444a2fd6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.377397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 328/658] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/role_assignments?user.id=15b5f371cd614e3bb246b015818a97f2&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:36.378222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.403815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.416424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.416997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.417281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.417551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.431311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.432976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:36.451018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:36.452046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:36.458492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:36.458492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:36.458492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:36.458492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:36.458492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 331/659] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:36.458492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.539407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864520b-0375-49e5-b48f-caca5a134b03 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.573015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.592058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.592734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.593026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.593309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.607764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.609262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.616057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13c605-3c9f-4637-8b33-a3c0aeda7d7c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:36.631114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:36.632352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:36.633242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:36.633242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:36.633242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:36.633242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:36.633242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 329/660] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:36.633514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.665228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.665941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.666145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.666432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.678806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.679958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:36.696103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:36.697234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:36.698143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:36.698143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:36.698143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:36.698143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:36.698143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 332/661] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:36.698424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.736450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.759208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc478c6-215f-4a5a-bc84-53c5bd04946b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.812870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.813893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.815082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.815601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.831800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:36.837285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:36.838260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 330/662] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:36.840525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:36.855694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccaf44-fb63-4143-a25a-7939719ae1d9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:36.886033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:36.897800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:36.898621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:36.899005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:36.899382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:36.900651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:36.900651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:36.902243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:36.920837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 333/663] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:36.920837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.005356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76d73-7ca1-4d74-bc78-3e894d487c84 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.044487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.048170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.048697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.048974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.049245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.049958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:37.050269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.051704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.057266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae0a2-d04f-438c-ac27-a5b88a057c9e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.066552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 331/664] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:37.067425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.117279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.118103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.118396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.118758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.119520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:37.119837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.121775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.136239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 334/665] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:37.137744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.190998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.204101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c9621-352c-447b-8284-83dcfeac8ad0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.255955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.257007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.257495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.257968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.258894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:37.259386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.261009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.267099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a68ad-d457-4edf-b413-7d94ad8c2b19 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.278162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 332/666] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:37.278162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.323339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.323950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.324763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.325038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.333539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.335006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:37.348836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:37.349913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 335/667] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:37.350929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.370021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.406226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e508-21bd-42b3-ad80-26a943b28890 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.465879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.466901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.467871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.468195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.476796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.478435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.482022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb618-9d42-4fda-a6dc-118a71a33ec1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:37.492537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:37.494400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 333/668] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:37.496030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.528956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.543025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.543726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.544018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.544348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.554207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.555720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:37.573776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:37.575392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 336/669] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:37.577080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.655907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39fdb0-3f9a-4e81-8833-d960ca9bbc58 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.704924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.722112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.722682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.722972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.723671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.737708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.740599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.747683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f148d0-498f-4eee-8215-e8c0683c79cc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:37.757812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:37.758785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:37.759686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:37.759686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:37.759686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:37.759686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:37.759686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 334/670] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:37.760035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.796083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.796724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.797006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.797280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:37.798213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:37.798521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:37.800013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:37.818695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 337/671] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:37.818695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:37.868424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.934212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ad0f7-72b6-49e6-b8ef-339d07c5a609 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.950509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd12463-8c5e-43fc-bd52-39b0fc7524be None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:37.994134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:37.997356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:37.998294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:37.999085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:37.999729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.001158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:38.001775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.002415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.003492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.003981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.004242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.004916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.005037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:38.005352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.006835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.028267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 335/672] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:38.029875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.038697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 338/673] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Aug 1 14:06:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:38.039639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.165720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa014e-d09f-47f8-b423-3bca742c3879 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.179066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a3e8-ccbc-425d-bf55-3f9eeffecb7f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.211501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:38.213121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.213744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.214019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.214288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.215135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=f3a7a9a0bda04d128a4f045fdb0bc80c) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:38.215490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.216956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.223823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.224537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.224903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.225109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.225828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:38.226137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.231792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 339/674] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/role_assignments?user.id=f3a7a9a0bda04d128a4f045fdb0bc80c&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:38.231984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.232084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.244702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 336/675] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:38.245346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.335041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:38.408596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593280bf-8e27-4a86-8588-41b8c2c91733 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.425675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a50329-604b-47a1-9042-a05b8ff71f63 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.490032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.490727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.491057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.491319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.492168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=ee96f24e85de4092b31081d44fe0a03d) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:38.492498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.492943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.493988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.494184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.494469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.495005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.502626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 340/676] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/role_assignments?user.id=ee96f24e85de4092b31081d44fe0a03d&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:38.503643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.602158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:38.620438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.622206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.670695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bcb2a-4f31-4e94-8f47-864581e4f7ef None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.691358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:38.716579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.717330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:38.717600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:38.717882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:38.727353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:38.729017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:38.747587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:38.749185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:38.750852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 341/677] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:38.759232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.818290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 337/678] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:38.820279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:38.955037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16b1da-9e62-479e-9efd-1ee0217495b3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.969982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:38.995926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd53e2-62aa-4445-b81c-386fbf4ae073 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:38.998729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:38.999622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.000083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.000551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.001551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:39.001858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.003318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328785f5-8380-4db3-8bb1-011c208bf95a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.041476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 342/679] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:39.041476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e37c6a-c540-4ea7-8ece-8b50c74e89b5 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.041476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e37c6a-c540-4ea7-8ece-8b50c74e89b5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.041476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e37c6a-c540-4ea7-8ece-8b50c74e89b5 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.041476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 343/680] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:39.044259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.044569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.044851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.071594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:39.072161 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.072943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:39.073558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.074637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.075131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.109378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.112347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.125739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:39.142475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:39.144130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:39.145878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:39.145878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:39.145878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:39.145878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:39.151551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 338/681] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:39.154241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.174473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 344/682] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:39.175512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.296164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753302b-e2df-4af6-8b02-6336b86f0a12 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.334238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfaac8-afd7-4e61-b3f3-c26d4ae1dc4a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.339366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.340703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.341014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.341254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.346348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:39.346348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.346348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.359605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 345/683] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:39.361106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.383763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.384557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.384845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.385186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.400165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:39.508952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda27c8-3bee-4e83-a18b-2f53cedbc61f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.537954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:39.550820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.551374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.551696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.551969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.564836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:39.570848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:39.571930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:39.573162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:39.575880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 346/684] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:39.576890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.631638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.635076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.685812 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:39.705047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373271b7-2d1c-4491-bb60-48aed1205539 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.747444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.748139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.748421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.749333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.757120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.758811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:39.771989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:39.773260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:39.774669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:39.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 347/685] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:39.779628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.794572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 339/686] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:39.796218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.874190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:39.920761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dece59f-3746-4832-8602-cdcdd98ba798 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.953923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389042a-1880-4c16-a75b-82bbf962270b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:39.962887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:39.963863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:39.964148 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:39.964421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:39.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:39.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:39.967954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:39.984888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 348/687] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:39.985739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:39.999918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:40.009038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.010157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.010795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.011338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.012499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:40.013038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.014981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.030833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 340/688] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:40.032380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.127980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81501dcf-a011-4b73-8615-30eef888ef2a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:40.172846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.173789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.174076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.174350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.188434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.190030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.196477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:40.200929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0b8a8-1ece-4dca-85ad-7934e9787392 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:40.209751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:40.210816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:40.211931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:40.211931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:40.211931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:40.211931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:40.213928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 349/689] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:40.215592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.252793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.253627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.254030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.254444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.271764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.273377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.290309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:40.300247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:40.301245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:40.302248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:40.302248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:40.302248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:40.302248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:40.302248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 341/690] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:40.302248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.363015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc566-ae82-4e7b-a1c2-e41b439bdf9c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:40.406592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.407549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.407975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.408394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.409652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=771f49a069014fdc956e073c2d0a0fdd) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:40.410116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.411985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.424067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 350/691] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/role_assignments?user.id=771f49a069014fdc956e073c2d0a0fdd&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:40.425294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.427835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429855-5b93-4ccb-928e-061e759d8c96 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:40.478916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.478916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.478916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.478916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.479389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:40.479389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.479389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.491521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 342/692] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:40.497473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.549599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:40.630820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc5eff-2f38-4b0e-a371-663ee29bc6d2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:40.700483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:40.726618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.729277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.729775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.730287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.731485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:40.731966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.734388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.759908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178ed05-3035-4139-a03a-21225e8604ce None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:40.762568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 351/693] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:40.770631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.830073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:40.831046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:40.831329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:40.831647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:40.842206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:40.844923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:40.859401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:40.860401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:40.861288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:40.861713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 343/694] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:40.867750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:40.968626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003ecd-eb18-47f6-8153-bd7b622c3031 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:41.022776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:41.023445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.023763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.024064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.035926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:41.038336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:41.057170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:41.059574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:41.061896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 352/695] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:41.067202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:41.137721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:41.139651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:41.211398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0064605-7a79-4825-8912-ab8a24c01b43 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:41.221476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade60181-9e6b-42e4-99c1-34e49a7b859f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:41.269025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:41.269935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.270431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.270974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.272538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:41.273054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:41.274906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:41.311502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 353/696] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:06:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:41.313689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:41.325785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:41.327334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.328088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.328684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.329665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:41.330162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:41.331995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:41.377543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 344/697] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:41.384630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:41.483367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25707663-9d95-408f-9468-dcc8a83513ce None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:41.485892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:41.529908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:41.530970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.531748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.532397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.576588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbd088-8f75-4016-8d4c-6164b123d830 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:41.619597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:41.621427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:41.622553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:41.628458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:41.629091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.629374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.629649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.630492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=3690b0c864cb48a4b55774bb8875b644) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:41.630805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:41.632836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2814a-b417-4539-93e5-2396bb8c9d3b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:41.651508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 345/698] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:41 2022] GET /identity/v3/role_assignments?user.id=3690b0c864cb48a4b55774bb8875b644&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:41.651508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fee1e8-cbb2-44fa-b27d-07e8cab2fa7e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.651508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fee1e8-cbb2-44fa-b27d-07e8cab2fa7e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.651508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fee1e8-cbb2-44fa-b27d-07e8cab2fa7e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.656025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 346/699] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:41.656025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb26f-93d6-4377-8194-b2ace17f1a75 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.656025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb26f-93d6-4377-8194-b2ace17f1a75 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.656025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb26f-93d6-4377-8194-b2ace17f1a75 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.660843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 347/700] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:41.662921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a25de-55a7-466b-b55d-8eca50cbefee None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.663377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a25de-55a7-466b-b55d-8eca50cbefee None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.663870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a25de-55a7-466b-b55d-8eca50cbefee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.692237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3e7a25de-55a7-466b-b55d-8eca50cbefee None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:41.693526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3e7a25de-55a7-466b-b55d-8eca50cbefee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:41.806106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 348/701] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:41.811535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 354/702] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:06:41.812352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:41.812757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:41.813487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:41.814238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:41.817178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:41.853992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:41.855948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:41.976524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 349/703] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:41.977241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:41.998406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ef488-34a1-4293-8883-d9d767c752cb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.057357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.059391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.060174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.060932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.065361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:42.081977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.085116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:42.114391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:42.116308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:42.118051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:42.118051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:42.118051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:42.118051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:42.118051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 355/704] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:42.118589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:42.156766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156ef3e-f14d-4e4f-ad73-9dcf700dd88f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.229696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.232319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.233383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.234108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.235752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.239297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:42.263547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:42.265647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:42.267449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:42.270768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:42.271516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 350/705] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Aug 1 14:06:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:42.274012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:42.321536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf887c-b309-4034-b1a7-c04f730a055f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.380229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.380830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.381294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.381696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.395838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.397465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:42.419119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:42.420316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:42.421395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:42.421395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:42.421395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:42.421395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:42.424943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 356/706] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:42.426160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:42.489105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5699b7c-8b34-4884-8e7e-8a0adb59772e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.526488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:42.550765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.552057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.552616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.553445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.554508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:42.556780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.558571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.580233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 351/707] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:42.581479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:42.612203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32383f-d633-440c-b95f-2a34667708ea None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.626019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:42.692919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.693997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.694492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.694865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.695745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:42.696178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.698034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.710159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 357/708] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Aug 1 14:06:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:42.712074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:42.759926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24623acb-11db-41d3-8a34-e88ed628451d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.834716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.837055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.838122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.838836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.841946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:42.843125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.847277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.849031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a995cf-21f7-4d48-a125-e80308ca234d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:42.872572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 352/709] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:42.873724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:42.903633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:42.904681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:42.905469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:42.905933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:42.921685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:42.927042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:42.936318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:42.966305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:42.968438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:42.970529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:42.983604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 358/710] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:42.984053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:43.048898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:43.070490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a401e1-b23a-4c3f-bcdf-69fff1467a74 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:43.144499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:43.145516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.146008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.146509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.148185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:43.148512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:43.148691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38daac-46c7-445b-98a9-a152a86283db None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:43.151151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:43.171366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 353/711] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Aug 1 14:06:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:43.172691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:43.227239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bc648-8a7f-4a59-ad02-fe4aa0b10cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:43.229573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc648-8a7f-4a59-ad02-fe4aa0b10cb6 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.231000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc648-8a7f-4a59-ad02-fe4aa0b10cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.231880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc648-8a7f-4a59-ad02-fe4aa0b10cb6 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.232763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bc648-8a7f-4a59-ad02-fe4aa0b10cb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:43.234473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bc648-8a7f-4a59-ad02-fe4aa0b10cb6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:43.272543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 359/712] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:06:42 2022] POST /identity/v3/users => generated 254 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:43.275073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686380f-9483-460d-aedc-39ffb6045b64 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.275655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686380f-9483-460d-aedc-39ffb6045b64 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.276178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686380f-9483-460d-aedc-39ffb6045b64 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.285904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 360/713] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:43.289280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.289758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.290210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.324089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:43.325295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:43.392988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62f904-50cd-4aae-9072-8ca3f4ae9594 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:43.409433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:43.461575 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 361/714] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:43.470790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:43.472508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.473603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.474400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.476319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:43.477266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:43.481194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:43.505905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:43.511247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 354/715] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:43.523139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:43.539425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:43.674842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfab7db-e0b9-4ae4-8cb5-61bd0d91205b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:43.677694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0047fd-f490-4eb6-b171-39a4b7df7162 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:43.717177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:43.717819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.718105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.718390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.719277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=15b5f371cd614e3bb246b015818a97f2) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:43.719639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:43.721390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:43.729289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 355/716] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:43 2022] GET /identity/v3/role_assignments?user.id=15b5f371cd614e3bb246b015818a97f2&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:43.731295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:43.732000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.732286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.732567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] PATH_INFO: `/v3/services/network` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.733246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:43.734714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:43.753496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:43.754612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:43.755889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 362/717] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:43 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:43.761681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:43.812408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:43.919494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094699-4f89-4663-998e-350605fe9408 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:43.961000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:43.961751 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:43.962139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:43.962447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:43.963153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] RBAC: Adding query filter params (name=network) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:43.963490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:43.965516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6f2e9-5ca5-4ada-93d3-37f42b9eb36e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:43.972597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 356/718] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:06:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:43.978955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:44.103532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:44.135189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1daa-d8f2-4acf-aa77-62276d6f610a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:44.187378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:44.188013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:44.188480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:44.188772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:44.189539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:44.189851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:44.191343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972f32-41ae-4877-8317-be8278e75ff3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:44.198508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 363/719] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:06:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:44.647823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337a9e-c5b4-45aa-8fc3-a855306460ec None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:44.648380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337a9e-c5b4-45aa-8fc3-a855306460ec None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:44.648380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337a9e-c5b4-45aa-8fc3-a855306460ec None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:44.650187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 357/720] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:44.656448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:44.656734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:44.657022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:44.689527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:44.690759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:44.805378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 364/721] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:44.842986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337a9e-c5b4-45aa-8fc3-a855306460ec None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:44.980650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:44.994599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337a9e-c5b4-45aa-8fc3-a855306460ec None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:45.036607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:45.037270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:45.037556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:45.037836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:45.049241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:45.054724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:45.055758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:45.056620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 358/722] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:45.064618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:45.221746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:45.286589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d659d-4bd9-4db3-8df2-089d7133d783 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:45.372040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:45.373106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:45.373610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:45.374111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:45.375625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:45.376188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:45.378781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:45.401676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 365/723] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:45.410096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:45.565432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7843d0-89d2-49db-acb4-db2dd3f5f276 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:45.605974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:45.606664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:45.607278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:45.607670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:45.619919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:45.622009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:45.638602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:45.639634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:45.640595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:45.640595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:45.640595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:45.640595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:45.640595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 359/724] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:45.646112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:45.934557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11ec53-1421-4fc1-99fd-4a3a1d114986 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:45.994697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:45.995356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:45.995698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:45.995979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:45.996724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:45.997035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:45.998491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d7922-9cda-4e9f-a341-76213a0d9b8c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:46.003571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:46.017226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 366/725] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:46.023014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:46.027700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:46.130465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94852-034b-4a78-9891-bbc91d684400 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.130933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94852-034b-4a78-9891-bbc91d684400 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.131229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94852-034b-4a78-9891-bbc91d684400 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.132667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 367/726] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:46.138192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.138493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.138765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.175911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518632f-3e8d-4ea6-9733-7b4b10a7699e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:46.179630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:46.180913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:46.245472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:46.246164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.246454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.246727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.261107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:46.267777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:46.291210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:46.293257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:46.295262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:46.296458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 360/727] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:46.307010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:46.340089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 368/728] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:46.418649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:46.461324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:46.527658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74a666-1ebb-478b-9bac-ebc3b5d14e94 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:46.571508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:46.571508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.571508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.571508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.572167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:46.572167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:46.572416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:46.600251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 361/729] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Aug 1 14:06:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:46.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:46.733685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7f1c7-37c1-435e-9f80-fd70c8eb969f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:46.767844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:46.772898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee36b02-9ff8-4aeb-9fab-2fedd1ff6bf9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:46.782525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79539b9-bd33-498f-90f0-71016794dc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:46.782845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79539b9-bd33-498f-90f0-71016794dc6c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.783219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79539b9-bd33-498f-90f0-71016794dc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.783533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79539b9-bd33-498f-90f0-71016794dc6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.865337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:46.870445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79539b9-bd33-498f-90f0-71016794dc6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:46.873948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79539b9-bd33-498f-90f0-71016794dc6c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:46.894124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934533dd-5318-47ce-805a-f3487a8bfc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:46.894774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934533dd-5318-47ce-805a-f3487a8bfc95 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.895061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934533dd-5318-47ce-805a-f3487a8bfc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.895339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934533dd-5318-47ce-805a-f3487a8bfc95 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.896085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934533dd-5318-47ce-805a-f3487a8bfc95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:46.897507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934533dd-5318-47ce-805a-f3487a8bfc95 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:46.943401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 369/730] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:06:46 2022] POST /identity/v3/services => generated 251 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:06:46.993304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65447b8f-e54e-4221-b835-a75e3a6a51aa None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:46.993644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65447b8f-e54e-4221-b835-a75e3a6a51aa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:46.993948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65447b8f-e54e-4221-b835-a75e3a6a51aa None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:46.996358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 370/731] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:47.005499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31adb8b-61ac-477f-81c9-bef5d0887d5c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.006008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31adb8b-61ac-477f-81c9-bef5d0887d5c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.006503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31adb8b-61ac-477f-81c9-bef5d0887d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.044546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b31adb8b-61ac-477f-81c9-bef5d0887d5c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:47.045709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b31adb8b-61ac-477f-81c9-bef5d0887d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:47.065370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 362/732] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:06:47.071030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855be04-fdd2-4b54-837e-058a3371d214 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.071344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855be04-fdd2-4b54-837e-058a3371d214 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.071639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855be04-fdd2-4b54-837e-058a3371d214 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.073278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 363/733] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:47.076499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855be04-fdd2-4b54-837e-058a3371d214 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.115953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:47.161807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 371/734] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:06:47.163239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a6493-c500-4fa1-9148-7d5479f1a245 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.164095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a6493-c500-4fa1-9148-7d5479f1a245 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.164095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a6493-c500-4fa1-9148-7d5479f1a245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.199034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bb4a6493-c500-4fa1-9148-7d5479f1a245 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:47.200293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bb4a6493-c500-4fa1-9148-7d5479f1a245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:47.227625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855be04-fdd2-4b54-837e-058a3371d214 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:47.272707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:47.273490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.274227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.274227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.294722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 372/735] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:47.294907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:47.295004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d9ff1-0e41-473a-97ca-bfd5c3cd1cd6 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.295107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:47.295203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d9ff1-0e41-473a-97ca-bfd5c3cd1cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.295203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d9ff1-0e41-473a-97ca-bfd5c3cd1cd6 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.296612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 373/736] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:47.298075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d9ff1-0e41-473a-97ca-bfd5c3cd1cd6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9c2b4c43-1d5e-4c3c-afa0-6c83105216f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:47.316924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:47.317997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:47.319539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:47.319539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:47.319539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:47.319539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:47.319539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 364/737] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:47.319539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.319896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.319896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.340613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:47.341838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:47.371071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:47.436872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 365/738] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:06:47.437779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.443941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:47.459796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d9ff1-0e41-473a-97ca-bfd5c3cd1cd6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:47.503834 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:47.504524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.504805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.505088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.505823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:47.507232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:47.526421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 374/739] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:06:47 2022] POST /identity/v3/services => generated 240 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:47.528208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.592479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec9171-07b9-46e0-bdb1-8f3ba4e39b8e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:47.663119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:47.664413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.664956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.665458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.666421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:47.667150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:47.669665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:47.681693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbea1b-3bc1-4cbe-9714-c15fa567d2c6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:47.683570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:47.691530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 366/740] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:47.694739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.727376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:47.728078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.728358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.728689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.745188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:47.751079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:47.752262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:47.753073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 375/741] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:47.753768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.790287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:47.868437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272355e6-20f7-4b2c-a3d0-152d0b32a268 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:47.892003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839783b9-9472-4615-9203-0bfe58c29a26 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:47.918473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:47.919339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.919765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.920119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.934019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:47.939103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:47.940358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:47.940791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:47.941702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:47.942533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:47.943570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:47.944433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:47.947680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 367/742] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:47.949571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:47.958992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:47.960502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:47.977712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:47.978725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:47.980508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:47.983562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 376/743] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:47.984783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.020287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.087783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1ebf-2da5-447e-9be3-7cc3a4f22040 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.112462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.132906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58807b-0e0a-4757-986d-9f4c5e061ed9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.133131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.133656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.133941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.134220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.134969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:48.135279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.137436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.150307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 368/744] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:48.151591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.186304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.186996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.187283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.187588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.188451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:48.188768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.190909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.210990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 377/745] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:48.212015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.296533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bc805-5785-4cbb-a171-1ea3aec83c60 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.349557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.352075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.352886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.353171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.353504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.354355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:48.354664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.356395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.375600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 369/746] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:48.375600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.381645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f5245-1879-4c3a-bc4e-f2c3188acf50 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.425140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.429385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.430371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.430754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.431129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.449416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.452189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:48.474465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:48.475729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:48.476655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:48.476655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:48.476655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:48.476655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:48.477336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 378/747] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:48.478375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.503998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a17e27-1ac2-48ad-b454-ba7140c73f40 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.562150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.563010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.563449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.563765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.579032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.581038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:48.603225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:48.604245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:48.605676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:48.605676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:48.605676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:48.605676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:48.608315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 370/748] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:48.609524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.616402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766174a-391c-4720-b07e-cda378906ed7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.670317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.671319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.671841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.672309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.673321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=3690b0c864cb48a4b55774bb8875b644) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:48.673851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.675609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.679924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.685021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 379/749] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/role_assignments?user.id=3690b0c864cb48a4b55774bb8875b644&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:48.686204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.775256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.788736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469768a3-1ca0-4f1d-a5c5-5f9e312ae73b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:48.848264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:48.849271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.849529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.853925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.853925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:48.853925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:48.853925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:48.873691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 371/750] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:48.875304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:48.931264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432c92-aa8b-45f7-a395-9ddb391687ec None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.008411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.009214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.009513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.009677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.010456 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:49.010765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.012436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.036447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ce83-7e8b-46f6-9032-3f8d5a6d3c4e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.042203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 380/751] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:49.047955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.083796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.085071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.085668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.086235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.096135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.099338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:49.116517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:49.117535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:06:49.118450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:49.122398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 372/752] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:06:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:49.134680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.175118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:49.206392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:49.315930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ede3c9-8ce6-44fd-a4b4-305aafe44994 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.351737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b93d-fffc-4879-bfa0-f382c237b8f2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.366023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.366796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.367076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.367550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.376366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.377944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7bcabe3b-c308-46e5-b1ee-462933b56409 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:49.393469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:49.394463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:49.395404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:49.396476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.396476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.396476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.396476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.396737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 381/753] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:49.396805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:49.396805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.396922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfb12c-bd10-4880-b957-7e9fb92cfb2a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.396922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfb12c-bd10-4880-b957-7e9fb92cfb2a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.397058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfb12c-bd10-4880-b957-7e9fb92cfb2a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.399642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccccb40-ce0e-4d76-94e2-4b86ece75e87 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.403199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 382/754] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:49.405037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfb12c-bd10-4880-b957-7e9fb92cfb2a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.426304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 373/755] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:49.427758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.428047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.428331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.455846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:49.458430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:49.560958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfb12c-bd10-4880-b957-7e9fb92cfb2a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.575017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 374/756] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:49.576095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.578587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:49.608834 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:49.631192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.631934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.632218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.632497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.633904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:49.634375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.636045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.662465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 383/757] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:49.663363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.749580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829b355-6640-4ea1-bfc9-dea20dab035d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.788341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.789119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.789412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.789695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.811595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfdee6-f39b-4101-82cf-264f0ff2e11b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.856296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.857452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.857878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.858318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.859111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.860927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.867999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.869473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.872116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:49.875230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:49.876603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:49.877579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:49.887612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 384/758] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:49.888880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.950269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.009418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 375/759] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:06:50.009418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.041209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6ff70-8ec4-4366-92d1-ef6ecabd5823 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.084338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.085263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.085546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.086002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.099605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.101218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5e363ae0-1659-41d6-b574-de0c96870101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:50.119373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:50.120626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:50.122007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:50.122007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:50.122007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:50.122007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:50.122007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 385/760] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:50.123489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6802f0b-5605-4ae4-adeb-39814996acc8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.123489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6802f0b-5605-4ae4-adeb-39814996acc8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.123489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6802f0b-5605-4ae4-adeb-39814996acc8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.127577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 386/761] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:50.127577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6802f0b-5605-4ae4-adeb-39814996acc8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.140619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bdfbf-8244-40d8-b25c-3e5aa0f92557 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.144395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.203835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.204460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.204742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.205041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.205733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:50.206036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.207771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.217886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 376/762] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:50.217886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.282081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.293161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6802f0b-5605-4ae4-adeb-39814996acc8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.348064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.349362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.349829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.350284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.367844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.370365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.373507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411540-5c61-4051-86da-e539d4e14002 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b0e84cd8-68d0-4540-9220-1bc33bc93c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:50.395435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:50.396419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:50.397579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:50.397579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:50.397579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:50.397579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:50.398608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 387/763] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:50.400215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.400694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.401254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.422847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:50.423990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:50.428934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.429571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.429860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.430136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.430846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:50.431156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.432607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.454282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 377/764] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:50.454282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.500845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.537609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 388/765] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:50.538854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.627894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a41fa-2853-412a-864b-a23a5b5322d7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.657258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.682529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.683167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.683472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.683832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.684464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:50.684771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.685345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a71e2-8691-4224-b399-bd7f9f6e8ecb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.686164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.697577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 378/766] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:50.697577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.731973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.732673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.732957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.733233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.733948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:50.734257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.735681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.756208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 389/767] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:50.756208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.810067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.821553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320edf3e-a1a9-4804-ad93-e102e46d9bef None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.867725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.868382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.868730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.869018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.877443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.878307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.890256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12fd5c-6035-4831-8f14-a11322ec0b59 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:50.891680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:50.895428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 379/768] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:50.899863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.936588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.937265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.937535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.937807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] PATH_INFO: `/v3/services/network` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.938476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.939935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.943346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:50.952916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:50.954258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:50.955313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:50.956426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 390/769] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:50.957336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.050604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b3b9f-7ea8-4929-8e1b-ed5775705dd4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.092369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017b07-e6bf-4e35-834e-4ca91cf7a2ec None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.097307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.097899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.098184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.098724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.099533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] RBAC: Adding query filter params (interface=public, service_id=08ce74828fd143fea2c74ce0577a9a92, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:51.099852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.101279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.109692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 380/770] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/endpoints?service_id=08ce74828fd143fea2c74ce0577a9a92&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:51.110644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.132736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.138753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.139842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.140319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.140766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.151958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.153460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:51.165947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:51.167316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:51.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 391/771] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:51.169022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.238223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.258676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de509-72f5-474a-9f18-a0a0dac71a4f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.296807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfec77-3794-46dc-82f1-eb6fa1ca493c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.302704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.303242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.303564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.303850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.304622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:51.304939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.306412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3544-689a-4f0c-a3b5-0fa14e62117c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.320614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 381/772] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:51.322777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb1fe2-b09a-494d-96ea-329fa781cdd3 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.323264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb1fe2-b09a-494d-96ea-329fa781cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.324366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb1fe2-b09a-494d-96ea-329fa781cdd3 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.326821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 382/773] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:51.328188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb1fe2-b09a-494d-96ea-329fa781cdd3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.350528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.351082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.351373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.351740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.353175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] RBAC: Adding query filter params (name=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:51.353606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.355430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.363180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 392/774] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:51.364181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.427950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.459887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb1fe2-b09a-494d-96ea-329fa781cdd3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.485570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f22a-4ae6-4c60-bcf7-11aa2edbe6ee None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.512610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.513239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.513519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.513791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.514461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.515006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.515882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.523187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 383/775] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:51.524287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.537702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.538208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.538583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.539042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.541391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:51.542033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.546254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa426c-8700-452e-808b-767d1abb90be None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.564571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 393/776] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:51.565972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.566315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.566623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.590925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:51.592897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:51.696386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22461-f458-4cf4-a943-1ca036036633 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.719510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 394/777] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:51.719510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.753911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.754744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.755437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.755437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/15b5f371cd614e3bb246b015818a97f2/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.758063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=15b5f371cd614e3bb246b015818a97f2, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.759793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.773968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.781092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 384/778] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:51 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/15b5f371cd614e3bb246b015818a97f2/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:51.782155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.887025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3acc17-3d38-48ac-8922-bfcfc1192310 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.910870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.940095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.940797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.941301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.941566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.942344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] RBAC: Adding query filter params (type=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:51.942779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:51.944524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:51.945077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d463d-62b4-467d-adce-57b5d878f360 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:51.953974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 395/779] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:51.954987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:51.996718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:51.997348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:51.997638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:51.997912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:51.998836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=8631c9e2a3934738999f0315215ba682) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:51.999151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:52.000727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:52.009603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 385/780] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/role_assignments?user.id=8631c9e2a3934738999f0315215ba682&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:52.010457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:52.155870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ea5d3-e1a7-484e-b021-9007b5c2816a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:52.169417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:52.231848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:52.232433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:52.232716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:52.232995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:52.248601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:52.250142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206f9f-3849-4758-845c-a73561cd82c9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:52.255573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:52.256647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:52.258158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 396/781] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:52.264443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:52.292519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:52.293507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:52.293507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:52.293896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:52.294639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] RBAC: Adding query filter params (interface=public, service_id=56fd39a1a2934f3699704d89ca79c85f, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:52.294953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:52.296659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:52.303609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 386/782] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:06:52 2022] GET /identity/v3/endpoints?service_id=56fd39a1a2934f3699704d89ca79c85f&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:52.309644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:52.340070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:52.463796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a4fb-3cc5-4424-bed2-4229054cb69e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:52.494682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:52.501055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc60f76-24be-4ed5-8ce7-d819412fdc07 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:52.506589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:52.507678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:52.508176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:52.508677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:52.510144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:52.512382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:52.519412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 387/783] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:06:52 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:52.571713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:52.572746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:52.573271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:52.574293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:52.575580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:52.576116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:52.578669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:52.596407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 397/784] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:52.604522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:52.812891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248248ff-e265-4c88-ab57-e830ab97388a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:52.855710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:52.856997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:52.857492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:52.857836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:52.875879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:52.878593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:52.913579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:52.914527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:52.915406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:52.915406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:52.915406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:52.915406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:52.917770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 388/785] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:52.927163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:53.153834 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:53.156928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b34567-3411-4686-b0bb-58bd85faf88e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:53.235579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:53.236271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:53.236558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:53.236831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:53.237533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:53.237862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:53.239855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:53.255189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 398/786] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:53.260733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:53.360863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:53.452998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423dfd-28f9-4c04-8146-b542904bf948 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:53.506012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:53.506770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:53.507059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:53.507337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:53.520660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:53.522132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:53.539903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:53.540893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:53.542991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 389/787] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:53.547495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:53.685032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3eca0-0838-4645-945d-5ec94df771f3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:53.725182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:53.725744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:53.726037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:53.726316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:53.727137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:53.727450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:53.728956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:53.750209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 399/788] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Aug 1 14:06:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:53.756805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:53.934016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7d224-4f1c-4f3f-998f-30b989e4367c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:53.935875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:53.980099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:53.981037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:53.981480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:53.981917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:53.998386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:54.000012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:54.017031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:54.018055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:54.019356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:54.019356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:54.019356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:54.019356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:54.019356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 390/789] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:54.028480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:54.063620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:54.155191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f443d1-a10b-4b8e-9cc9-d45da7c6e6d2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:54.194360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:54.195026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.195256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.195574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.196357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:54.196689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:54.198231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab5725-dbbf-458a-b18f-659b5550bcd4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:54.212911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 400/790] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:54.220392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:54.289943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30042b2-1937-439e-909e-b88dfd878ccb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.290242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30042b2-1937-439e-909e-b88dfd878ccb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.290526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30042b2-1937-439e-909e-b88dfd878ccb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.292298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 401/791] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:54.304956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c7b3e-eb93-493b-9d41-4763635fcc26 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.305248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c7b3e-eb93-493b-9d41-4763635fcc26 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.305531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c7b3e-eb93-493b-9d41-4763635fcc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.343908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f07c7b3e-eb93-493b-9d41-4763635fcc26 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:54.346371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f07c7b3e-eb93-493b-9d41-4763635fcc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:54.454390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd499908-60bd-4a01-b428-834dbaaeb1b1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:54.480579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 402/792] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:54.482838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea8f1-9ce4-41a6-b1db-fe289022ebfe None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.483335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea8f1-9ce4-41a6-b1db-fe289022ebfe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.483851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea8f1-9ce4-41a6-b1db-fe289022ebfe None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.486006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 403/793] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:54.492127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.492412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.492691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.511515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:54.512354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.512828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.513224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.521576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:54.522437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:54.522827 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:54.524238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:54.525986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:54.539029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:54.540088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:54.541002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 391/794] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:54.541486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:54.561221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:54.618458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 404/795] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:06:54.619372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:54.704213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedbd5a-447a-4ac1-9006-c0f0433db280 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:54.752478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d208b-ee85-4736-95f0-fb0933b5e431 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:54.770378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:54.771845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.773754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.774589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.776163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:54.779193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:54.794712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:54.795608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.796064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.796512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.797934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:54.798511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:54.799866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:54.801555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:06:54.803076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:54.804555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:54.807816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 392/796] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Aug 1 14:06:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:54.810354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:54.813103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:54.825249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 405/797] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:54.826757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:54.946771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0cbc7-bd9c-45f6-bb16-461f5af4ee62 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:54.965998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:54.991913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:54.992953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:54.993606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:54.993933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:54.994828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:54.995144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:54.996681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:55.002709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 406/798] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Aug 1 14:06:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:55.003202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.013659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443cdb3-9158-4f2e-873e-b36f1a171694 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.073902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.073902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.073902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.073902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.080878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:55.087089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:55.094047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:55.095200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:55.096056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 393/799] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:55.097230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.152334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d91966-f656-4c07-bcff-a061c5c323b3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.206085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fce75-835f-4bb5-bf5e-ce60ee036e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.207264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fce75-835f-4bb5-bf5e-ce60ee036e2f None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.207543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fce75-835f-4bb5-bf5e-ce60ee036e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.208044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fce75-835f-4bb5-bf5e-ce60ee036e2f None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/3690b0c864cb48a4b55774bb8875b644/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.211785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fce75-835f-4bb5-bf5e-ce60ee036e2f None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=3690b0c864cb48a4b55774bb8875b644, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:55.215003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fce75-835f-4bb5-bf5e-ce60ee036e2f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:55.247599 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 407/800] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:55 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/3690b0c864cb48a4b55774bb8875b644/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:55.249505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9a5bd-4162-41fa-be54-43eea001666b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.249798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9a5bd-4162-41fa-be54-43eea001666b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.250079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9a5bd-4162-41fa-be54-43eea001666b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.256383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 408/801] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:55.256383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9a5bd-4162-41fa-be54-43eea001666b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.273514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792eb7f-99f2-4db5-a37e-c77d3699f7d7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.338894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.341458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.342323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.342823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.344110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:55.344673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:55.347221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eeaf3-3a8e-48df-9459-c9e74028156d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:55.356712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 394/802] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Aug 1 14:06:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:55.359043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73975f2-cc3b-4d48-aa6e-79fb231c29c5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.359378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73975f2-cc3b-4d48-aa6e-79fb231c29c5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.360230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73975f2-cc3b-4d48-aa6e-79fb231c29c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.371959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:55.386450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d73975f2-cc3b-4d48-aa6e-79fb231c29c5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:55.387771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d73975f2-cc3b-4d48-aa6e-79fb231c29c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:55.441125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9a5bd-4162-41fa-be54-43eea001666b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.474878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:55.490820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.491446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.491755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.492072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.492750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:55.493058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:55.498950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 395/803] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:55.499520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37066c61-03f8-42ca-941f-ef00b8e39e68 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:55.499607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c6cd1-9ffe-49ae-9715-f817f42085a1 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.499607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c6cd1-9ffe-49ae-9715-f817f42085a1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.499607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c6cd1-9ffe-49ae-9715-f817f42085a1 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.499607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 396/804] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:55.499607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c6cd1-9ffe-49ae-9715-f817f42085a1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.513849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 409/805] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:55.515260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.515571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.515841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.538035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:55.539317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:55.630281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 410/806] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:55.633051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.650944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c6cd1-9ffe-49ae-9715-f817f42085a1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.717158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:55.728662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.729335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.729620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.729948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.730942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:55.732073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:55.762182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 397/807] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:06:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:55.763257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.787000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d72e4f-42d2-4e99-9fc3-1174c9d1f922 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.809546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:55.850035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.850661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.851237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.851721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.874598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:55.877134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:55.902124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:55.903125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:55.904033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:55.904033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:55.904033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:55.904033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:55.910778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 411/808] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:55.912275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:55.930299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed62eb-5604-4fb7-a52a-c79e31b8e13a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:55.974894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:55.975755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:55.976147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:55.976527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:55.991047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:55.998445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:56.000503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 398/809] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:06:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:56.002356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.092715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.116638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d162-d673-4547-99e4-63cbbd6d048b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.168759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.169502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.169885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.170247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54e23-bc7a-40e1-a2a4-0ac40661469a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.170365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.171068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.172929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:56.183951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:56.185031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:06:56.185831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:56.186563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 412/810] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:55 2022] GET /identity/v3/services/network => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:56.188856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.217235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.217868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.218157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.218435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.219278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.219906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:56.220318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.221873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.238964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 399/811] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:56.239989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.307409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723093-cb64-4c4e-b485-59fef65a89d4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.349426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.350073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.350355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.350637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.351501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:56.351909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.353466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.363389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dd94a-f249-424e-a482-d613e5f7b194 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.371322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 413/812] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:56.372514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.374629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.406831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.407419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.407745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.408026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.408727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] RBAC: Adding query filter params (name=network) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:56.409038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.411295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.416856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 400/813] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:56.418379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.492547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.506940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb41c2-dcd2-4ea6-be37-ad942e2abe71 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.545960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.546610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.546896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.547174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.547542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879ea4-005e-4ff8-9903-2c72a2faf590 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.556292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.557905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:56.573180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:56.574841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:06:56.576034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:56.576910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 414/814] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:56.577785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.601508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.602498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.602970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.603431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.626147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.627815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:56.650404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:56.651373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:56.652963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:56.652963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:56.652963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:56.652963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:56.657728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 401/815] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:56.661319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.701589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.733908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bd76b-1e1b-48dc-b154-d9d39c9dac13 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.785539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.787433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.787714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.788100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.789216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] RBAC: Adding query filter params (type=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:56.789634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.792058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.799922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 415/816] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:56.801719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.807084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82ce87-d4e1-44a9-802c-5d194a1f7977 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.826303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.854143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.855401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.855897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.856443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.857556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:56.858090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.859811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:56.882569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 402/817] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:56.883796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:56.947289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3034b-d96b-49c8-8ed6-3b2c68ea5a6d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:56.983047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:56.989162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:56.990857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:56.991340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:56.991823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:56.992725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:56.993249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:56.997641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.016903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 416/818] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:57.016903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:57.021466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b1c1d-70a9-426d-9918-64e980440916 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:57.066244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:57.067313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.067927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.068415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.069534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:57.071321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.100165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 403/819] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:06:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:06:57.101490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:57.114525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:57.221362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea42f4b-7e88-4dea-bcaf-df7e7b5bee54 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:57.278044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2479c06-c91d-4ad5-9049-b253031d70ea None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:57.343231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:57.343803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.344157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.344420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.346948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:57.348823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.349651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.350904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.355012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:57.365991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:57.368765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:57.386632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:57.387989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:06:57.388939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:57.396085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 404/820] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:06:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:57.398321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:57.471075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:57.472703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.536904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f1f15-775b-43e5-a727-75aea05b8225 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:57.568468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:57.591118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:57.591853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.592070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.592349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.593176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:57.593540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:57.594964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.616463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 405/821] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Aug 1 14:06:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:57.618176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 417/822] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:06:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:06:57.624795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:57.625014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:57.689257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:57.746767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee3bf8-c17f-4d4a-9aa2-b93657667665 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:57.765993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5462a-2ffc-44b1-88a8-fe4c0734d1be None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:57.793634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:57.794498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.794880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.795258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.809200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:57.811069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.827535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:57.828631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.829459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.829960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8ffe520d-4357-4a32-823e-e0de71250a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:57.838083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:57.840133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:57.841836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:57.841836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:57.841836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:57.841836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:57.845575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 406/823] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:57.846858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955844-fe0f-4603-94ba-52ed1cecc121 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.847152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955844-fe0f-4603-94ba-52ed1cecc121 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.847432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955844-fe0f-4603-94ba-52ed1cecc121 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.849403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 407/824] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:57.850181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955844-fe0f-4603-94ba-52ed1cecc121 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:57.861244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:57.862915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-652f5bdd-bed3-4ba7-8ad1-9ab7205a2925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:57.890315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:57.891280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:57.892201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:57.892201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:57.892201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:57.892201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:57.892201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 418/825] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:57.892825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:57.893112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:57.893382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:57.916567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:57.918533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:57.941846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.014196 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 419/826] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:58.015422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.017399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955844-fe0f-4603-94ba-52ed1cecc121 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.031876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.082166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.083206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.083742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.084228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.085668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:58.086038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.088712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.120504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 408/827] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:58.120504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.182423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa916e-ceac-4f24-b05a-cf4f44871938 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.242571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.244113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.245533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.246045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.246976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:58.248290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.251503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.267077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 420/828] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:58.268477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.291100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.352929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0ec21-0f52-4b9e-aba9-01dfbdb25f05 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.393843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.394267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.394610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.394951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.409205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:06:58.409471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58a6ed-c5f8-4da4-b45b-419d3b537300 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:06:58.415718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:58.416720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:06:58.417574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:58.418694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 409/829] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:58.419195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.451805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.459019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.459715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.459997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.460278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.468367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.469709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-920002b5-d3e1-47c2-aefa-21d88305cb92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:58.482041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:58.483113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:58.484200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:58.487636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 421/830] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:58.489492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480fabe-11ea-4ddc-a028-a0b672f8538f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.489800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480fabe-11ea-4ddc-a028-a0b672f8538f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.490100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480fabe-11ea-4ddc-a028-a0b672f8538f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.491534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 422/831] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:58.492317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480fabe-11ea-4ddc-a028-a0b672f8538f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.549463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d53b-d61c-4945-b899-346ccdf02f36 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.587884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.594064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.594722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.595009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.595288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.603107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.604666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:06:58.617308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:58.618267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:06:58.619246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:58.619711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480fabe-11ea-4ddc-a028-a0b672f8538f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.619785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 410/832] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:58.619785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.668776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.669402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.669690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.669961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.670696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:58.671005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.672475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccba8e-0228-4b20-9168-07404baae84a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.681867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 423/833] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:58.683702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.684082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.685993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.706699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:58.707985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:58.732924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.751628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c6f9-b86f-47e6-b17c-e72f8fd374bd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.791944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.792595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.792872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.793143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.804029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 424/834] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:06:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:06:58.804290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:58.804290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.804469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.804544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.819200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 411/835] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:58.820711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.871047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:58.915095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eac80-a4b1-455e-9218-a55bb89c3291 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.945469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38df0b-0cb9-4043-8987-96ce4fa56d46 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:58.955014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.955680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.955957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.956231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:58.957096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:58.957446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:58.958982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:58.975738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 425/836] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:58.977335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:58.987819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:58.988780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:58.989202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:58.989609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.007676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.009188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.012737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:06:59.033357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:59.034479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:59.041294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:06:59.041294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:06:59.041294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:06:59.041294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:59.041294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 412/837] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:59.041294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.103283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1fc63-c5f4-497d-a42c-29eda9d46cfc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.131901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:59.145144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.145808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.146088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.146358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.146993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.148409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:59.157956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:59.159796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:06:59.160954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 426/838] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:06:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:06:59.161853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.167701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ff693-eecd-4ba6-b635-17017bd55b5b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.212854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.213530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.213820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.214153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/8631c9e2a3934738999f0315215ba682/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.233818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=8631c9e2a3934738999f0315215ba682, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.234770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.252946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 413/839] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:06:59 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/8631c9e2a3934738999f0315215ba682/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:06:59.253805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.315173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4563ad-c7a5-4f79-a31b-9d607a38f20d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.322353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:59.365493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.366673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.367248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.367809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.369459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=15b5f371cd614e3bb246b015818a97f2) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:59.370781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.373520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.391889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 427/840] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:06:59 2022] GET /identity/v3/role_assignments?user.id=15b5f371cd614e3bb246b015818a97f2&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:59.393298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.418719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f1aa5-0035-4298-bac7-19ac7bd01f2f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.477832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.478479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.478771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.479052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.479880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:59.480195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.481691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.493868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:59.499937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 414/841] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:59.508052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.569100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313d342-1009-4c83-88d1-cb04ca8fe51c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.615374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.616142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.616401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.616659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.617313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:59.618245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.621083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.626446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 428/842] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:06:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:59.637903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.681612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88cc9b-038a-42c1-902a-681fb75da072 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.692140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:59.732380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.733596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.734084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.734556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.748492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.751587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.767713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43834732-972f-47c3-aa1b-7734900b77bc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:06:59.777382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:06:59.778605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:06:59.779736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:06:59.783786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 415/843] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:06:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:06:59.790890 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:59.792833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:59.816075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.816985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.817488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.817909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.818831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:59.819298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.820941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:59.827428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 429/844] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:06:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:06:59.952507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389b55b-adc2-4766-ba71-c2a625597abe None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:59.993136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:59.993768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:59.994053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:59.994335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:59.995352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:06:59.996220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:59.997760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:00.019172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 416/845] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Aug 1 14:06:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:00.024543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:00.142506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7bd4-be30-4385-ba74-ca5e77df99ca None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:00.181687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:00.182317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.182626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.183480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.239146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:00.263231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:00.265057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:00.378230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 430/846] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:00.383725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:00.420501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:00.510099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a8c1e-5a95-4a59-b4a8-69943200d724 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:00.553848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:00.554510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.554830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.555110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.566863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:00.569047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:00.586456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:00.587540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:00.588430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:00.588430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:00.588430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:00.588430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:00.593453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 417/847] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:00.596657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:00.700252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:00.727430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a624fef-9c97-4c69-84ac-50e3d6a6776b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:00.763225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:00.763802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.764085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.764358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.765050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:00.765356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:00.766727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:00.779846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 431/848] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:00.785587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:00.882121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:00.910450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8180-a0fa-4d69-9457-a6aef4c2adc4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:00.947652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:00.948943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.949222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.949529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.957251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:00.959057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:00.971504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:00.972472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:00.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:00.975625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 418/849] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:00.979735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:01.107492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ca19a-032f-499b-8bcf-27f22401288c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:01.151005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:01.151925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.152212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.152485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.153254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:01.153560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:01.155150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4c644-c6d6-4197-83ae-88fd8258d822 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:01.169875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 432/850] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:01.175974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:01.297056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63475f-3d9e-4dbb-a97f-33abafec3b7e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:01.337330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:01.337985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.338291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.338619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.339487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=3690b0c864cb48a4b55774bb8875b644) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:01.339802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:01.341256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3f3f9-0bc3-4e9c-b7f0-1c0452c43e25 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:01.350159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:01.359109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 419/851] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:01 2022] GET /identity/v3/role_assignments?user.id=3690b0c864cb48a4b55774bb8875b644&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:01.616096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafbd1c-131a-4aa0-a18a-ed55da53f957 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.616620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafbd1c-131a-4aa0-a18a-ed55da53f957 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.616620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafbd1c-131a-4aa0-a18a-ed55da53f957 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.618161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 433/852] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:01.624415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32074f4-25dc-47c6-b0a8-cc5839f63645 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.625312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32074f4-25dc-47c6-b0a8-cc5839f63645 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.626174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32074f4-25dc-47c6-b0a8-cc5839f63645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.648003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b32074f4-25dc-47c6-b0a8-cc5839f63645 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:01.649549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b32074f4-25dc-47c6-b0a8-cc5839f63645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:01.736820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 420/853] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:01.757687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafbd1c-131a-4aa0-a18a-ed55da53f957 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:01.761013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:01.824435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d6a0-acad-43ca-8082-adbe89c10fc9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.824960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d6a0-acad-43ca-8082-adbe89c10fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.824960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d6a0-acad-43ca-8082-adbe89c10fc9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.826625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 421/854] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:01.832122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.832409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.832687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.857565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:01.858874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:01.889815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafbd1c-131a-4aa0-a18a-ed55da53f957 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:01.939750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 422/855] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:01.944652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:01.945310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.945592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.945866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.956470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:01.958440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:01.964454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:01.965648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:01.966460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 434/856] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:07:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:01.972097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.023815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:02.075615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:02.086425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee1661-aba7-4c9f-83ce-f5507c03ce88 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.123969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f1b09-a44e-464f-8c85-81bdae6a45fc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.133861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.134524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.134887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.135208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.136613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.138113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:02.152714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:02.154734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:02.156899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 423/857] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Aug 1 14:07:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:02.167517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.177618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.178253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.178535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.178808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.179577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:02.180038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.181401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.191213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 435/858] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:07:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:02.199488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.303546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905994e3-cc01-4b7a-bf10-dc22ee0289ae None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.332672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:02.346596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a96e-ff10-4b53-9b67-56dabd55d4ee None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.348640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.349427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.349910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.350350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.352242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:02.352242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.353786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e9896-b8be-4f03-b92a-83357efb8028 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.364064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 424/859] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:02.366521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5487c2-c98b-48b4-abd8-1657a1b5a676 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.366819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5487c2-c98b-48b4-abd8-1657a1b5a676 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.367472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5487c2-c98b-48b4-abd8-1657a1b5a676 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.369046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 425/860] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:02.372435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5487c2-c98b-48b4-abd8-1657a1b5a676 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.393742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:02.407600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.408441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.408854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.409326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.422705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.424330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9f59dd5b-e93a-498c-9f0a-dab3e341f33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:02.457033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:02.458786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:02.461066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:02.461066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:02.461066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:02.461066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:02.461066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 436/861] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:02.461608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.462092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.462375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.488135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:02.489646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:02.560234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5487c2-c98b-48b4-abd8-1657a1b5a676 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.603689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 437/862] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:02.603689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.624877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.625750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.626247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.626683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.627980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:02.628514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.631380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.639501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 426/863] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:02.643097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.714589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:02.777526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6894-d887-40a9-9bdf-49431707038b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.794199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c633e-2868-4ff5-82a9-1b4803621853 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:02.811576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:02.818872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.819563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.819838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.820116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.820890 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.822238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.846800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 427/864] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:02 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:02.878840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:02.879514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.880087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.880424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.881200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:02.881561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:02.883180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:02.903055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 438/865] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:02.905538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:03.036135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28aa853-5c69-450b-8016-414cc95f4cce None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:03.053584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:03.088811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:03.090172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:03.090549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:03.090921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:03.101180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:03.102814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:03.115947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:03.117198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:03.118783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:03.121994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 428/866] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:07:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:03.128687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:03.309253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1dda-2cd4-42e5-ad11-f2accf23efcf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:03.371845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:03.372793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:03.373216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:03.373638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:03.374961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:03.375429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:03.377888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c9946-522c-4ed4-a41f-29fc8e9f340e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:03.410642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 439/867] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Aug 1 14:07:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:03.416978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:03.562769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59b97-6ed2-47e3-bfa3-81df50511d81 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:03.602528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:03.603623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:03.604114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:03.604621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:03.650880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:03.682974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:03.684944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:03.732435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596be2-58b3-4c4b-a3de-1e3ba1abcece None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:03.733105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596be2-58b3-4c4b-a3de-1e3ba1abcece None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:03.734089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596be2-58b3-4c4b-a3de-1e3ba1abcece None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:03.736656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 440/868] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:03.741942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:03.742235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:03.742518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:03.762901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:03.764157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:03.782408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:03.841034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 429/869] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:03 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:03.844099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:03.871938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 441/870] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:03.894416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:03.939531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:03.989725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9408067-d1d0-41e9-9cf6-97485d828e1e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.033551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.034308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.034701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.035140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.050351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.053986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.065323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0a01-2fea-4c0f-bcf4-c2c948c34166 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:04.082379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:04.083417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:04.084324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:04.084324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:04.084324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:04.084324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:04.087850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 430/871] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:04.093424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:04.117614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.118635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.119142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.119617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.121642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.123545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:04.140824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:04.141888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:04.142700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:04.146485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 442/872] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:04.154625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:04.232533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75a80-ad5e-4c12-b262-c1622f428909 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.233060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:04.265543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:04.278789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.280176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.280661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.281473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.282692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:04.283197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.285482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.303510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 431/873] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:04.306040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7107977-d566-4191-ad11-cdc4adce9a3d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.311025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:04.350905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.352185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.352471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.352743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.353446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:04.353811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.355931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.366531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 443/874] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:04.367777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:04.444719 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dbbc6-3f0a-47fd-8826-d04bfd029b42 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.467244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:04.486907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece493-837d-4345-bf77-19316189d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.488011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece493-837d-4345-bf77-19316189d13d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.488505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece493-837d-4345-bf77-19316189d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.488989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece493-837d-4345-bf77-19316189d13d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.497471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece493-837d-4345-bf77-19316189d13d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.499361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece493-837d-4345-bf77-19316189d13d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.500401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef782780-15d3-4fcb-8140-386fdc75f6e4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-20ece493-837d-4345-bf77-19316189d13d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:04.514521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:04.515515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:04.516401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:04.525688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 432/875] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:04.527572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece493-837d-4345-bf77-19316189d13d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:04.541125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.541768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.542584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.542859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.543781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:04.544237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.545652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.557180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 444/876] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:04.598907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:04.690688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece493-837d-4345-bf77-19316189d13d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.731891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.732532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.732885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.733092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.733878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:04.734187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.735690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2998b-4b2b-41e0-8782-3497b1328837 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.752767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 433/877] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:04.760725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:04.905156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e62822-b6b4-435c-89df-da62a9bcc903 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:04.944717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:04.945662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:04.946129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:04.946589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:04.947665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=8631c9e2a3934738999f0315215ba682) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:04.948217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:04.949894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af986d6a-6a06-44ba-8cd3-010c3546e33c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:04.957565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 445/878] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:04 2022] GET /identity/v3/role_assignments?user.id=8631c9e2a3934738999f0315215ba682&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:05.071846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d2c3c-5e2a-4412-9db5-561c55ae0491 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.072405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d2c3c-5e2a-4412-9db5-561c55ae0491 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.072520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d2c3c-5e2a-4412-9db5-561c55ae0491 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.075093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 434/879] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:05.081307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.081630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.081873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.099358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:05.107225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:05.108894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:05.129494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa7cd6-58c4-4016-bcc2-5108b40b3866 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.129806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa7cd6-58c4-4016-bcc2-5108b40b3866 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.130038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa7cd6-58c4-4016-bcc2-5108b40b3866 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.131819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 435/880] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:05.138572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.138888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.139175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.166875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:05.168285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:05.206477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 446/881] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:05.238597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:05.259076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 436/882] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:05.288536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:05.329726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:05.403991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65874a-0b39-498f-9eec-fdaf100f6e21 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:05.425005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd046c2-2a34-4e3f-8631-89b219d587d9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:05.444243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:05.445016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.445298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.445633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.446340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:05.447758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:05.453255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:05.470736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:05.471970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.472617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.473507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.474424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:05.476026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:05.485222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 447/883] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:05 2022] POST /identity/v3/services => generated 269 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:05.493974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:05.495954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:05.497513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 437/884] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:07:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:05.505867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:05.651942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a309c8-3b15-4ce2-a1b9-ff26acfab968 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:05.703788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:05.704813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.705335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.705860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.706948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:05.707258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:05.708750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:05.715098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 448/885] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:07:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:05.720933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:05.847178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:05.851758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382870c0-cd61-4d34-b760-b2156d9af0f3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:05.890772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:05.891406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:05.891724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:05.891999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:05.892699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:05.893009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:05.894423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:05.905195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 438/886] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:07:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:05.906830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:05.987352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:06.044606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d70f6-ede6-4ce6-9cd1-f7e82aefb6d8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:06.091734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:06.092380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:06.092665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:06.092942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:06.093774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] RBAC: Adding query filter params (interface=public, service_id=5f1079b97a4a4697b2bfa0860fcfbbbb, region_id=RegionOne) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:06.094094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:06.095705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5cc70-99ec-44be-8180-6c53f78378eb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:06.106785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 449/887] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:07:05 2022] GET /identity/v3/endpoints?service_id=5f1079b97a4a4697b2bfa0860fcfbbbb&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:06.115197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:06.235655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f424-f770-4a88-bdc1-fddd685e5291 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:06.284185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4da8c-aa5e-4fe1-9eca-674b3472ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:06.285238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4da8c-aa5e-4fe1-9eca-674b3472ed76 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:06.285731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4da8c-aa5e-4fe1-9eca-674b3472ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:06.286216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4da8c-aa5e-4fe1-9eca-674b3472ed76 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:06.287088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4da8c-aa5e-4fe1-9eca-674b3472ed76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:06.289403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4da8c-aa5e-4fe1-9eca-674b3472ed76 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:06.307609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 439/888] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:07:06 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:07:06.778376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c865-47a2-4228-9142-67d9d960038f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:06.778752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c865-47a2-4228-9142-67d9d960038f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:06.778931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c865-47a2-4228-9142-67d9d960038f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:06.780634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 450/889] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:06.786520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833050a5-c79f-48c4-9994-a804f79d596f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:06.786827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833050a5-c79f-48c4-9994-a804f79d596f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:06.787191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833050a5-c79f-48c4-9994-a804f79d596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:06.814826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-833050a5-c79f-48c4-9994-a804f79d596f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:06.816459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-833050a5-c79f-48c4-9994-a804f79d596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:06.842050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:06.903384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 440/890] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:06.926678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c865-47a2-4228-9142-67d9d960038f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:07.068895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c865-47a2-4228-9142-67d9d960038f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:07.094292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:07.124030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2c64b-6a6d-4799-9715-b6adfe874389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:07.125684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c64b-6a6d-4799-9715-b6adfe874389 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:07.126270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c64b-6a6d-4799-9715-b6adfe874389 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:07.127369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c64b-6a6d-4799-9715-b6adfe874389 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:07.129334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2c64b-6a6d-4799-9715-b6adfe874389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:07.132220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2c64b-6a6d-4799-9715-b6adfe874389 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:07.161120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 451/891] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:06 2022] POST /identity/v3/services => generated 241 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:07.783370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044032ca-493c-481b-ae92-ea480ff8dde1 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:07.783931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044032ca-493c-481b-ae92-ea480ff8dde1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:07.784286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044032ca-493c-481b-ae92-ea480ff8dde1 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:07.787066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 441/892] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:07.794444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2662d8-f120-426a-b8be-94bd62531f39 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:07.795046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2662d8-f120-426a-b8be-94bd62531f39 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:07.795553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2662d8-f120-426a-b8be-94bd62531f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:07.820605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1c2662d8-f120-426a-b8be-94bd62531f39 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:07.822305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1c2662d8-f120-426a-b8be-94bd62531f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:07.929490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 452/893] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:07.958206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044032ca-493c-481b-ae92-ea480ff8dde1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.108619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044032ca-493c-481b-ae92-ea480ff8dde1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.114740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478242-b56a-42f9-8e82-89b4c243ff2f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.115116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478242-b56a-42f9-8e82-89b4c243ff2f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.115336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478242-b56a-42f9-8e82-89b4c243ff2f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.116898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 453/894] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:08.126661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.127305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.127833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.148335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:08.150439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:08.163246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:08.164419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.164931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.165404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.182847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:08.187966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:08.189594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:08.190408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 442/895] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Aug 1 14:07:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:08.197354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.232657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:08.275837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 454/896] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:08.294351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.335492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:08.338023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652437ff-39ae-49ce-8f4d-ff98b115a86e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.379479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:08.380291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.380717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.381153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.382023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:08.382334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:08.383940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:08.398484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 443/897] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:08.399626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.428356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690e484-373a-40e6-b7f8-982ee7e7b2b6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.484329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:08.485363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.485843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.486306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.487336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:08.489024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:08.493709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:08.504290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:08.505892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:08.507173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 455/898] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:08.515968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.528974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a9158-e739-41fa-aaa0-37907b4d53e3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.583641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:08.584254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.584655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.584804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.597448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:08.599271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-92dde796-2f11-4d1f-88af-571ab36eca64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:08.615136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:08.616989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 444/899] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f01900-e165-45d4-b986-c0b3a2f636a8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f01900-e165-45d4-b986-c0b3a2f636a8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.618776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f01900-e165-45d4-b986-c0b3a2f636a8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.624682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 445/900] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:08.625588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f01900-e165-45d4-b986-c0b3a2f636a8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.669376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:08.672857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91ff0f-f28a-49b0-b721-4fb9831fef1f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.721149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:08.721814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.722490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.722490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.723025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:08.723695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:08.725033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade0b3-d960-4514-bd2c-f971e3f95c2a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:08.736770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 456/901] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:08.736770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b827cce-a824-4626-984b-3de988866052 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.736770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b827cce-a824-4626-984b-3de988866052 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.736770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b827cce-a824-4626-984b-3de988866052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.761877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1b827cce-a824-4626-984b-3de988866052 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:08.763204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1b827cce-a824-4626-984b-3de988866052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:08.764655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f01900-e165-45d4-b986-c0b3a2f636a8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.800814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:08.809263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:08.809897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:08.810178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:08.810453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:08.811281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:08.811742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:08.813252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:08.829579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 446/902] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:08.830450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.857143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 457/903] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:08.858886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b827cce-a824-4626-984b-3de988866052 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:08.959971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d677a8c-3346-4693-9c1d-3a756836e4aa None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:08.992380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.002960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b827cce-a824-4626-984b-3de988866052 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.008337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.009243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.009676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.010106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.011258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:09.011839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.014548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.020789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 447/904] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:09.021851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.052485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.053164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.053399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.053664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.063092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.064526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:09.079350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:09.080460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 458/905] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:07:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:09.086709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.106988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.166090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527940a-59e8-4700-94d8-953c913b1a2d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.215346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.216038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.216332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.216591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.217236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.218626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.226360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f60aa-3fb1-4ae7-aea5-b3836dfa3d7a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:09.228483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:09.230283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:09.231939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 448/906] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:09.233533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.279097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.279938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.280220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.280499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.281505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] RBAC: Adding query filter params (interface=public, service_id=11fb11844a5841039d5301d2fba192ff, region_id=RegionOne) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:09.281893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.283307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.289468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 459/907] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/endpoints?service_id=11fb11844a5841039d5301d2fba192ff&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:09.290103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.315360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.393243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddaf9b-309e-4d18-9503-ab95ccd12904 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.413134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.417585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a014b-9971-483a-bc40-b4adb38d22c1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.434127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.434905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.435354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.435806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.436878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:09.437239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.439082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8e24f-8614-4ecd-a28e-6481d4cfebc2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.456647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.457277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.457559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.457833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.458511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:09.458823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.460233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.465642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 460/908] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:09.466476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.467624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 449/909] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:09.469009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f0254-e88b-4f30-84c2-f04d2a496ba2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.469297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f0254-e88b-4f30-84c2-f04d2a496ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.469575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f0254-e88b-4f30-84c2-f04d2a496ba2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.471190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 450/910] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:09.472030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f0254-e88b-4f30-84c2-f04d2a496ba2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.586573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.590008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f0254-e88b-4f30-84c2-f04d2a496ba2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.613126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6583238-1436-4ac0-970f-48253aceeba9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.630252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.630886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.631169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.631442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.632154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:09.632458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.633832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.639791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 451/911] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:09.643062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.656027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e1f3a-46e8-43ef-aa3f-0464463c58bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.656650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1f3a-46e8-43ef-aa3f-0464463c58bd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.656936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1f3a-46e8-43ef-aa3f-0464463c58bd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.657214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1f3a-46e8-43ef-aa3f-0464463c58bd None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.657880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e1f3a-46e8-43ef-aa3f-0464463c58bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.659338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e1f3a-46e8-43ef-aa3f-0464463c58bd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.676788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 461/912] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:07:09.676788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.676788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.676788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.698405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:09.699797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:09.705128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.782021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2cad9-2671-407e-b292-1e3684fbd664 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.804415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 462/913] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:09.805439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.843942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.845233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.845507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.845776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.929613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:09.941390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.944000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.957628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840d83-dc86-4b50-903a-f296c07044a7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.993832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.008023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.008836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.009182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.009464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.010251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.011918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.040851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 463/914] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:07:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:10.041599 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.108990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 452/915] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:10.109928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.228656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a164c-c0d5-4b8d-9270-2d1d51a3f94e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.257592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bea4d-85ea-4e95-96b2-da4f9f4b03b3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.266164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.281201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.282583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.283539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.284063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.284752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.286157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:10.297514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:10.298576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:10.299385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:10.304393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 464/916] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:10.305062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.307354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.307966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.308245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.308734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.325167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.327721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.329304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:10.346888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:10.348922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:10.350702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:10.350702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:10.350702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:10.350702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:10.350702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 453/917] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:10.354038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.434711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f47553-d08f-44ff-b111-1e0ddb9de3fa None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.464652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c339e8-7437-4c50-9eb9-3b8d070a7819 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.477865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.479162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.479796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.480281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.481317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:10.481835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.483599 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.495095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 465/918] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:10.496431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.504361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.505272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.505711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.506132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.507286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:10.507784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.510166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.530544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 454/919] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:10.537404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.570524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.606254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.623441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9328c-71c7-4aa0-9007-922cbea1daa7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.669062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544b0f6-6c0b-466e-9e97-b3ebe3563d74 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.674563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.675199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.675504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.675785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.676558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:10.676926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.679031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.687069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 466/920] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:10.692780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.716388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.717492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.717992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.718496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.732168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.734813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:10.756405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:10.759730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:10.763160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:10.771977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 455/921] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:10.771977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.814149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef789fc-801f-413d-806d-ffed3a307161 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.837130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.855224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:10.855904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:10.856164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:10.856445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:10.857216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] RBAC: Adding query filter params (interface=public, service_id=dd259b0b47294444ad278b7e9d0c5c6b, region_id=RegionOne) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:10.857532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:10.858994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:10.866327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 467/922] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/endpoints?service_id=dd259b0b47294444ad278b7e9d0c5c6b&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:10.874807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:10.950638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91239b-27c9-4a1e-b227-3e24e292cc9e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:10.968032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:10.997494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bf7bd-b1a6-431d-b862-7ecdf997d851 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:11.006426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:11.007073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:11.007356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:11.007674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:11.008459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:11.008780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:11.011591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:11.034272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 456/923] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:11.038639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:11.047283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1ac66-4ce3-49f3-a88d-f0841852cb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:11.048093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1ac66-4ce3-49f3-a88d-f0841852cb08 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:11.048219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1ac66-4ce3-49f3-a88d-f0841852cb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:11.048489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1ac66-4ce3-49f3-a88d-f0841852cb08 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:11.049153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1ac66-4ce3-49f3-a88d-f0841852cb08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:11.050655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1ac66-4ce3-49f3-a88d-f0841852cb08 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:11.063807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 468/924] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:07:10 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:07:11.197514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66a26-e36d-481a-981f-ebc7b940fdb4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:11.246317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:11.247404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:11.247925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:11.248397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:11.249532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] RBAC: Adding query filter params (role.id=3a2df3b6646746de9b17517a42ce6916, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=8631c9e2a3934738999f0315215ba682) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:11.250128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:11.252085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ae822-12f7-4bbc-b339-71c37ff10a8c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:11.267579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 457/925] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:11 2022] GET /identity/v3/role_assignments?user.id=8631c9e2a3934738999f0315215ba682&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=3a2df3b6646746de9b17517a42ce6916 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:11.977571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644ad1-8765-40f1-85a3-17d615a3b994 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:11.977840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644ad1-8765-40f1-85a3-17d615a3b994 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:11.978244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644ad1-8765-40f1-85a3-17d615a3b994 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:11.980215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 469/926] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:11.985582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:11.986339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:11.986339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.007386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:12.008675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:12.046279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:12.102974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 458/927] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:12.128743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644ad1-8765-40f1-85a3-17d615a3b994 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:12.192067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:12.255214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644ad1-8765-40f1-85a3-17d615a3b994 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:12.294397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:12.295140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.295432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.295737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.296374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:12.297752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-91db9d42-19d8-442c-b158-e24505b448ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:12.308018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:12.309008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:12.309830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 470/928] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Aug 1 14:07:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:12.316894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:12.367093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272c43-8d01-459b-894c-49acdc29c4d1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.367388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272c43-8d01-459b-894c-49acdc29c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.367688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272c43-8d01-459b-894c-49acdc29c4d1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.369077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 471/929] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:12.374917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.375199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.375501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.399794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:12.402043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:12.440562 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d6d12-37d6-4edd-b9b2-875787103582 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:12.501139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 472/930] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:12.506754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:12.507797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.508070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.508504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.509679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:12.510179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:12.515809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:12.521602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 459/931] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:12.527847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:12.532551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:12.576939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:12.591450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:12.678885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224111a5-0c2f-41d9-b972-a1f28b023051 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:12.684923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9aae8-ba28-4d5f-9b2a-2e0369d4af13 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:12.723568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:12.724632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.725356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.725809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.726818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:12.728625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:12.732214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING keystone.server.flask.common [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 14:07:12.738652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:12.739672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.740189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.740666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.741578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:12.742094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:12.743733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:12.749577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 473/932] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:12.759745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:12.759937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.sql.core [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 14:07:12.759937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 14:07:12.759937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [parameters: {'id': 'eeb5315e8b884f77b7ac2e270f79b105', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 14:07:12.759937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80525) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eeb5315e8b884f77b7ac2e270f79b105.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eeb5315e8b884f77b7ac2e270f79b105. Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:07:12.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result.read() Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 14:07:12.777593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application yield resource Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise self.value Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application session.commit() Aug 01 14:07:12.778528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.session.flush() Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application compat.raise_( Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 14:07:12.779441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 14:07:12.780351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result.read() Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:07:12.781175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [parameters: {'id': 'eeb5315e8b884f77b7ac2e270f79b105', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:12.782091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:12.782840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eeb5315e8b884f77b7ac2e270f79b105. Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:12.784199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 460/933] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Aug 1 14:07:12 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:12.790121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:12.896169 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a4b1f-35fe-43cc-8509-2430e2f564c9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:12.905723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:12.922060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:12.943335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:12.944174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:12.944561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:12.944936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:12.945734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834400fc-2f64-478f-b06d-070dd2814fb3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:12.945933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:12.947416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:12.987703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 474/934] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:07:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:13.016448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:13.017811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.018268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.018694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.028329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:13.030300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ea53b8e2-71e2-434f-8edc-ea699c7f174f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:13.046502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:13.047654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:13.048763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:13.051088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 461/935] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:13.058591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:13.196928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf3650-3272-4e49-9e98-bc886204c9c5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:13.236524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:13.237276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.237604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.237878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.238893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:13.239404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:13.242109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e2709-3ef4-484c-b781-87f1537a11de None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:13.268441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 475/936] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:13.587019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deff3d-5b14-437b-9f20-855c74e84427 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.587289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deff3d-5b14-437b-9f20-855c74e84427 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.587605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deff3d-5b14-437b-9f20-855c74e84427 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.590044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 462/937] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:13.595915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.596185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.596463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.600689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a73d32-4f2a-458d-a1be-598fd4c3b0a6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.600991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a73d32-4f2a-458d-a1be-598fd4c3b0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.601239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a73d32-4f2a-458d-a1be-598fd4c3b0a6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.603031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 463/938] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:13.609261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.610099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.610406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.621591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:13.622846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:13.632185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:13.633886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:13.645918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:13.726572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 476/939] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:13.738220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 464/940] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:13.752208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:13.760802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:13.863999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:13.876190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e0e2a-1c76-4c01-9eb3-8673cd88d277 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:13.877964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd02a47-9ded-49a0-a1dc-db3a7c536abc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:13.924229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:13.925035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.925417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.925699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.926548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:13.927922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:13.928172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:13.928584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.928865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.929137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:13.939966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:13.941075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:13.941936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 477/941] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:13.944435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:13.948358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:13.953394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:13.955551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:13.956552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 465/942] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Aug 1 14:07:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:13.965604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:13.981019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:14.083115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ed0ac-acf1-493d-85d6-e653db719a89 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.114125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0b3ff-c4c2-4105-b0b3-23c88f6325ab None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.142004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.143075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.143623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.144132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.145070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:14.145606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.147281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.154213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 478/943] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:14.160892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:14.166405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:14.173519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.174547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.175054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.175608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.176582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:14.177110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.178755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.193568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 466/944] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:07:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:14.194920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:14.265620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:14.294505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a268b3-310f-49de-b304-01b0fc7a7a17 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.312640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40a9d7-2e52-4d78-9157-4f733008e037 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.338923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.339657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.339941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.340217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.340990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:14.341304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.343908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.355788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 479/945] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:14.356740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:14.359931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.360970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.361478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.361971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.377272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.379295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:14.396816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:14.397804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:14.398699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:14.398699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:14.398699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:14.398699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:14.405921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 467/946] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:14.407373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:14.462883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:14.498135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4edd75-a560-4211-9d58-3388d1142db7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.529009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46390328-3a12-448a-b659-56c7cfc88ce2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.542568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.543302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.543892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.543892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.544615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.545999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.569310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:14.583784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 480/947] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:07:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:14.590026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.591239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.591964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.592480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.593473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:14.594063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.595994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.614556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 468/948] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:14.622274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:14.762412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99b8c9-f601-4ef9-a515-16e8de40e655 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.812815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:14.813426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:14.813706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:14.813977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:14.824583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:14.826061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:14.846269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:14.847281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:14.848537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 481/949] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:07:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:14.855493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:14.978035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39c351-be8c-4cb1-a786-79b0cfb5111b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:14.980774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:15.046120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.047186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.047757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.048258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.050304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:15.051171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.053010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.075557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 469/950] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Aug 1 14:07:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:15.084836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.168992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:15.220113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c029c3a-362e-4712-994d-ecc70baa2632 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:15.268586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.269264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.269552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.269901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.284482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.286007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-8eaaa714-7d57-46ee-9571-900c3748f890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:15.304605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:15.305547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:15.306448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:15.306448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:15.306448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:15.306448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:15.306448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 482/951] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:15.313341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.435204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d82c5d-eca5-4c49-8e1e-3b46eadd1011 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.435582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d82c5d-eca5-4c49-8e1e-3b46eadd1011 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.435822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d82c5d-eca5-4c49-8e1e-3b46eadd1011 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.438695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 483/952] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:15.443272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.443591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.443909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.447095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16159b5-ac10-4824-ab1a-71222f81b14d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:15.469450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:15.471833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:15.493081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.494908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.495399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.495911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.497167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:15.497482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.499238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.519368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 470/953] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:15.526615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.560192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:15.610858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 484/954] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:15.639992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.675788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497c03e-9ca2-4dd6-8054-e6b0d96cd368 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:15.715956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:15.727959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.728610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.728929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.729210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.736785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.738267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d7956d78-f945-4b4c-b542-5fedc18683c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:15.751728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:15.752862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:15.754123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 471/955] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:15.754657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5384479-6868-429f-863a-b3facc3e9ba8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.754657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5384479-6868-429f-863a-b3facc3e9ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.755229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5384479-6868-429f-863a-b3facc3e9ba8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.756184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 472/956] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:15.761777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5384479-6868-429f-863a-b3facc3e9ba8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.798160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55ecee-5227-47cb-94b1-79798c7d35f3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:15.841307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.841969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.842268 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.842593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.843139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.844553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d41aa009-4f52-4011-923b-35037296ed5b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:15.854953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:15.856110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:15.857631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 485/957] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:15.859317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.859317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.859317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.884367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:15.888558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:15.889806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:15.910470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5384479-6868-429f-863a-b3facc3e9ba8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:15.951841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.952858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.953378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.953756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.954746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:15.955010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.956628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.973558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 473/958] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:15.974613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.985173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 486/959] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:15.985943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:16.037202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:16.137797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fda98-18f1-4ce6-b33f-dbb76d715d78 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:16.159083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf771d-745a-4c65-9474-27a23ef22b8b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:16.182925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:16.184002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:16.184442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:16.184861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/8631c9e2a3934738999f0315215ba682/roles/3a2df3b6646746de9b17517a42ce6916` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:16.187924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=8631c9e2a3934738999f0315215ba682, role_id=3a2df3b6646746de9b17517a42ce6916)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:16.189705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:16.206788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:16.207518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:16.207823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:16.208101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:16.219810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 487/960] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:07:15 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/8631c9e2a3934738999f0315215ba682/roles/3a2df3b6646746de9b17517a42ce6916 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:07:16.220025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:16.220025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:16.220230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:16.220326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:16.227935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 474/961] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:16.236518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:16.241015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:16.358529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:16.369121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ec59c-8eee-4165-9bbe-a967be61527c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:16.408476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d14c-a7f5-43de-89ce-834e0582c88e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:16.427561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:16.428360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:16.428747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:16.429149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:16.430383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:16.432897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:16.437095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING keystone.server.flask.common [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 14:07:16.456144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.sql.core [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 14:07:16.456144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 14:07:16.456144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [parameters: {'user_id': '79eaddb3233647e8aed7dc30aeb29c6c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 14:07:16.456144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80524) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 01 14:07:16.462919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:07:16.464020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application yield resource Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise self.value Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application session.commit() Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 14:07:16.465165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.flush() Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application compat.raise_( Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 14:07:16.466093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:07:16.467053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:07:16.472918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [parameters: {'user_id': '79eaddb3233647e8aed7dc30aeb29c6c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:16.474027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 14:07:16.475409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:16.477301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 488/962] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Aug 1 14:07:16 2022] POST /identity/v3/users => generated 160 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:16.478091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:16.478091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:16.478091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:16.478091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:16.478388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:16.478388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:16.478388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:16.478569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:16.485339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 475/963] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:16.616806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4b88-ddd7-4f2b-8744-c653748e8841 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:16.664004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:16.664683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:16.664970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:16.665247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:16.675425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:16.677357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b665c2bb-222a-4493-87c6-d69e10208247 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:16.692103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:16.693886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:16.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 489/964] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:07:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:16.699743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:16.802699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:16.846992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7a956-a5dc-41f6-b955-3a652df8fd84 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:16.895297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:16.896358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:16.896824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:16.897303 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:16.898732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:16.899252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:16.902187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228d8a-1d85-4288-ad0a-a1173d54f0d0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:16.950507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 476/965] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Aug 1 14:07:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:17.038562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aacbe-bc8b-4d60-912a-7b33ce70899e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:17.038909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aacbe-bc8b-4d60-912a-7b33ce70899e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:17.039302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aacbe-bc8b-4d60-912a-7b33ce70899e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:17.045756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 490/966] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:17.051729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:17.052052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:17.052285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:17.080774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:17.082690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:17.177953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 477/967] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:17.201695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705aacbe-bc8b-4d60-912a-7b33ce70899e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:17.233757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:17.363101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705aacbe-bc8b-4d60-912a-7b33ce70899e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:17.410721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:17.411574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:17.412139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:17.412464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:17.413340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:17.415000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:17.426476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:17.428098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:17.429064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 491/968] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:17.437264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:17.542840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:17.599145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c39b51-6106-43c7-9344-7d724243ab5d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:17.647921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:17.648658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:17.648946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:17.649287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:17.650127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:17.650446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:17.652097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:17.667725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 478/969] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:17.673226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:17.788224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:17.812118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b02e0-e53f-4111-9792-ce2532050906 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:17.854146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:17.854697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:17.855053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:17.855339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:17.856068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:17.856455 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:17.857880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:17.868614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 492/970] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:17.869736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:17.995426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba07db8-a43e-4e76-9bb4-09f4bcfee0c7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:18.023980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:18.043440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:18.044278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.044386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.044797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:18.045433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] RBAC: Adding query filter params (interface=admin, service_id=dd259b0b47294444ad278b7e9d0c5c6b, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:18.045748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:18.047852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a32b00-9e65-4612-a89e-b52c3e7d8230 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:18.056051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 479/971] 10.222.0.17 () {56 vars in 1249 bytes} [Mon Aug 1 14:07:17 2022] GET /identity/v3/endpoints?service_id=dd259b0b47294444ad278b7e9d0c5c6b&interface=admin®ion_id=RegionOne => generated 193 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:18.061743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:18.185633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a61514-b84b-4c4d-a503-c693687b138b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:18.226166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943e7e0-7520-4e61-a5d8-dcf01fc7cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:18.226883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943e7e0-7520-4e61-a5d8-dcf01fc7cd5d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.227183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943e7e0-7520-4e61-a5d8-dcf01fc7cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.227497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943e7e0-7520-4e61-a5d8-dcf01fc7cd5d None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:18.228242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943e7e0-7520-4e61-a5d8-dcf01fc7cd5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:18.229904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943e7e0-7520-4e61-a5d8-dcf01fc7cd5d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:18.243501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 493/972] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:07:18 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:07:18.629858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc159d-2b8b-4757-b4a6-d9a391df7a33 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.630782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc159d-2b8b-4757-b4a6-d9a391df7a33 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.630782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc159d-2b8b-4757-b4a6-d9a391df7a33 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:18.631876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 480/973] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:18.638627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.639347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.639347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:18.671897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:18.674370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:18.679442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:18.755211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1ddb6-eeb9-47a9-b7f8-68794aec9c3b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.755538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1ddb6-eeb9-47a9-b7f8-68794aec9c3b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.755818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1ddb6-eeb9-47a9-b7f8-68794aec9c3b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:18.757192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 481/974] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:18.763299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.763631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.763910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:18.778087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 494/975] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:18.784781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:18.785995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:18.796903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:18.866318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 482/976] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:18.889974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:18.940131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b991da-922d-44cf-96da-3ef9fd2bc031 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:18.985289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:18.985938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:18.986224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:18.986608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:19.000008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:19.003673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:19.012284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:19.013547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:19.014669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:19.023095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 495/977] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Aug 1 14:07:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:19.024948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:19.029433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc4a03-cb1c-472f-88d3-42b03277ae08 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:19.047317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:19.071858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:19.073272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:19.073611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:19.073926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:19.074757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:19.076262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:19.095807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 483/978] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:18 2022] POST /identity/v3/services => generated 242 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:07:19.167059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a111200-5a0e-4053-9b98-c75a78cc0fa0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:19.220383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:19.221594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:19.222092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:19.222548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:19.223790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:19.224320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:19.225928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:19.245205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 496/979] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:07:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:19.247166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:19.373586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4176f3-a056-4816-9144-e0be49ee2737 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:19.419229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:19.420104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:19.420399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:19.420683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:19.436921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:19.439755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:19.459538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:19.460481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:19.461435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:19.461435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:19.461435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:19.461435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:19.461435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 484/980] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:19.468274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:19.549947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:19.621992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d6d7-50f0-4e35-98a7-0415a6fb4448 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:19.681245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:19.681914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:19.682232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:19.682443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:19.683187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:19.683751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:19.685580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:19.712024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 497/981] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:19.714506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:19.753673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:19.883079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be768549-2ad4-4f9f-90d9-07b6c5806c7c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:19.946198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:19.946859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:19.947190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:19.948180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:19.958384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:19.959964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:19.973731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:19.974785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:19.976176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 485/982] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:07:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:19.981655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:20.139514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a1375-8655-481e-96bc-be929bcf1994 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:20.165997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:20.203418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:20.204194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.204675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.204905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.205847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:20.206194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:20.208008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5c5d-ee9d-4375-bb24-e38762f24f8a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:20.236304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 498/983] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Aug 1 14:07:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:20.242079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:20.375225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:20.395717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdb7cd-4863-4d47-be8d-0e4bce5f5d08 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:20.442244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:20.442871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.443175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.443487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.455604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9bde-1514-4751-9c75-0895c0685976 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.455604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9bde-1514-4751-9c75-0895c0685976 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.455604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9bde-1514-4751-9c75-0895c0685976 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.455604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 499/984] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:20.460309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac12708-76a2-4298-8617-c06373389195 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.460696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac12708-76a2-4298-8617-c06373389195 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.460849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac12708-76a2-4298-8617-c06373389195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.481123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9ac12708-76a2-4298-8617-c06373389195 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:20.482302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9ac12708-76a2-4298-8617-c06373389195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:20.567785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 500/985] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:20.575825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:20.577505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:20.587063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac12708-76a2-4298-8617-c06373389195 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:20.696442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:20.716324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 486/986] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:20 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:20.722066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:20.724888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac12708-76a2-4298-8617-c06373389195 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:20.736852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:20.768694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:20.769423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.769740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.769963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.770647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:20.772126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:20.783197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:20.785425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:20.786269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 501/987] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Aug 1 14:07:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:20.790651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:20.876087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6affe98-851c-4b2a-923e-8062dd20210b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:20.929347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e78b04-aab3-452c-b7ea-fb7cfdd44e8e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:20.937324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:20.937991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.938287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.938635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.950463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:20.951975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:20.969523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:20.970458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:20.971369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:20.971369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:20.971369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:20.971369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:20.971369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 487/988] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:20.975531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:20.976660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:20.977168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:20.977638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:20.977849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:20.978680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:20.979177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:20.981878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:20.997623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 502/989] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:20.998736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.034181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:21.042345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:21.124037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d453a1-2ce4-4172-bd0e-6682cd818f59 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.131259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d43c0-fcf7-4933-a5f0-8cf99359dc35 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.166591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.167307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.168029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.168142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.168740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:21.169122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.170535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.176139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 503/990] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Aug 1 14:07:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:21.181602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.182367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.182759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.183044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.183369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.184096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:21.184407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.185829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.204994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 488/991] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:21.206416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.332927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:21.337293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69d66-656f-4eac-9fe0-e3623b434133 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.347747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:21.352589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d906-be64-4269-a87c-de54a8e11662 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.378579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c86f8f-a792-4750-a3c5-b009f98d55a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.379342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c86f8f-a792-4750-a3c5-b009f98d55a4 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.379698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c86f8f-a792-4750-a3c5-b009f98d55a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.380078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c86f8f-a792-4750-a3c5-b009f98d55a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.380707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c86f8f-a792-4750-a3c5-b009f98d55a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.382087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c86f8f-a792-4750-a3c5-b009f98d55a4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.392893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.393553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.393830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.394157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.400111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 504/992] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:07:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:07:21.401507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281147-c2e4-4a96-8a31-41fbd782569b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.401637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.401801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281147-c2e4-4a96-8a31-41fbd782569b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.402076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281147-c2e4-4a96-8a31-41fbd782569b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.403153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.403423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 505/993] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:21.409514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.409807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.410605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:21.417938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:21.418984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:21.419906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 489/994] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:21.425782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.435969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:21.437353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:21.560117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 506/995] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:21.583206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.597077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5913-fa34-4992-a242-5725195ab12b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.652262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.652987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.653274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.653551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.654332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:21.654646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.656284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.671097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 490/996] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:21.676983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.700723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:21.703132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:21.728303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1391-af5a-4030-992b-c3bd24650aa4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.770868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.771592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.771830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.772100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.772748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.774161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:21.784191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:21.786292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:21.787929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 507/997] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:21.792878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:21.816224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664effa3-726f-4498-abfa-42f7032b7014 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.857295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.858020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.858330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.858662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.859613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] RBAC: Adding query filter params (role.id=3a2df3b6646746de9b17517a42ce6916, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=8631c9e2a3934738999f0315215ba682) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:21.859932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.861461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.879497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 491/998] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:21 2022] GET /identity/v3/role_assignments?user.id=8631c9e2a3934738999f0315215ba682&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=3a2df3b6646746de9b17517a42ce6916 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:21.931911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217adcdc-c6bd-4502-b92c-b4d29e0a4897 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:21.984529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:21.985178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:21.985458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:21.985735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:21.986440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:21.986745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:21.988586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:21.995045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 508/999] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:22.001097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:22.108792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:22.133655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f5d9-56e2-48a2-a61a-c9d20fd1f4e9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:22.176187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:22.176828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:22.177115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:22.177394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:22.178104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:22.178418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:22.179872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:22.186406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 492/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:22.192736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:22.212806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:22.324014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9096-ab85-4b33-943a-0c899b92ff07 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:22.362280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:22.362803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:22.363152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:22.363439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:22.364282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] RBAC: Adding query filter params (interface=public, service_id=ee311df694df4aadad2e64a638d1b4e6, region_id=RegionOne) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:22.364609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:22.366021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fa7b3-48b8-42d2-bd9f-d15e616bff2f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:22.377413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 509/1001] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:07:22 2022] GET /identity/v3/endpoints?service_id=ee311df694df4aadad2e64a638d1b4e6&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:22.378655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:22.497570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78bd39-8ece-4b74-a229-7cc15d031b5e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:22.537617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac973b6-b1bb-4df2-bdbb-0a2e459bd94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:22.538645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac973b6-b1bb-4df2-bdbb-0a2e459bd94a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:22.539132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac973b6-b1bb-4df2-bdbb-0a2e459bd94a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:22.539630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac973b6-b1bb-4df2-bdbb-0a2e459bd94a None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:22.540764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac973b6-b1bb-4df2-bdbb-0a2e459bd94a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:22.543758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac973b6-b1bb-4df2-bdbb-0a2e459bd94a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:22.559550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 493/1002] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:07:22 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:07:23.680978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736051c-c009-41d7-95c6-4d307e4b0011 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:23.681794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736051c-c009-41d7-95c6-4d307e4b0011 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:23.681794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736051c-c009-41d7-95c6-4d307e4b0011 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:23.682992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 510/1003] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:23.691386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:23.691999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:23.692330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:23.717336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:23.718555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:23.722147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:23.809380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 494/1004] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:23.833153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736051c-c009-41d7-95c6-4d307e4b0011 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:23.941989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:23.975043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736051c-c009-41d7-95c6-4d307e4b0011 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:24.019913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce704-2ae9-45da-9f05-3dbecd39d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:24.020622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce704-2ae9-45da-9f05-3dbecd39d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.020909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce704-2ae9-45da-9f05-3dbecd39d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.021182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce704-2ae9-45da-9f05-3dbecd39d4e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.022471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce704-2ae9-45da-9f05-3dbecd39d4e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:24.023951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce704-2ae9-45da-9f05-3dbecd39d4e4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:24.026602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 511/1005] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:24.032901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:24.137823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86965e-59a9-4249-8c46-933bc14f4984 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.138177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86965e-59a9-4249-8c46-933bc14f4984 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.138488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86965e-59a9-4249-8c46-933bc14f4984 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.143087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 512/1006] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:24.146740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.147264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.147749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.167415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:24.168942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:24.169514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e512aa6-9bc1-4c21-a228-3e0b3e27f349 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:24.211789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:24.212337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.212623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.212932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.213634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:24.215113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:24.227763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:24.251498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 495/1007] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:24 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:07:24.261628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 513/1008] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:24.287352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:24.436468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087969-4cc6-47ad-bbad-6c775c95c13f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:24.490697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:24.491427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.491819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.492142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] PATH_INFO: `/v3/services/image` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.492796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:24.494295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:24.510502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:24.511834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:24.512672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 496/1009] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Aug 1 14:07:24 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:24.520931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:24.625904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:24.667514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e836bd3-316b-4a3a-8643-92e0e0925910 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:24.709124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:24.709831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.710019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.710351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.710953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] RBAC: Adding query filter params (name=image) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:24.711275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:24.712743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0f27f-379c-47fa-99b0-3d191237a8db None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:24.720831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 514/1010] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:24.726823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:24.779180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859d941-5573-4131-b308-cc00ea67c24d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.779557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859d941-5573-4131-b308-cc00ea67c24d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.779755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859d941-5573-4131-b308-cc00ea67c24d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.781388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 515/1011] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:24.788994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.789278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.789557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.803737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:24.815909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:24.818156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:24.859038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6f06-40e2-4914-9910-a06fe864c878 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:24.900906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:24.901534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:24.901844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:24.902095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:24.902790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] RBAC: Adding query filter params (type=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:24.903185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:24.904624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:24.909606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 516/1012] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:24.919493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 497/1013] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:24.936698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:25.028337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:25.077802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1497-ef15-424f-8dff-bfc91f5d6b90 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:25.134807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:25.134807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:25.134807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:25.134807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:25.135814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:25.139501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:25.146791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:25.147869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:25.148707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 517/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:25.155543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:25.299047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8070458-ba8e-4900-830e-af1253f9b632 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:25.332107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:25.350865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:25.351548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:25.351869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:25.352166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:25.352805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:25.353113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:25.355069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:25.361561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 498/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:25.367326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:25.492525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393077-7ac9-4425-9ee4-541e9ad45e56 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:25.535448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:25.543605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:25.544283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:25.544567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:25.544846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:25.545544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] RBAC: Adding query filter params (type=compute) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:25.545854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:25.547263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8591e-4e76-4e00-a795-d8bfb45c95ab None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:25.554088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 518/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Aug 1 14:07:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:25.561089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:25.714255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae605f87-842d-4c25-a68f-b8b171120faf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:25.757913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc66c3-ab7a-432d-996f-125e5804f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:25.758807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc66c3-ab7a-432d-996f-125e5804f591 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:25.758807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc66c3-ab7a-432d-996f-125e5804f591 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:25.759077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc66c3-ab7a-432d-996f-125e5804f591 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:25.759913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc66c3-ab7a-432d-996f-125e5804f591 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:25.761349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc66c3-ab7a-432d-996f-125e5804f591 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:25.793946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 499/1017] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:07:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:07:26.489290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618f52d-a4b7-4a6f-9eb9-c3710cb199aa None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:26.489867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618f52d-a4b7-4a6f-9eb9-c3710cb199aa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:26.489867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618f52d-a4b7-4a6f-9eb9-c3710cb199aa None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:26.491377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 519/1018] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:26.497343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:26.497616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:26.497916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:26.518351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:26.519678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:26.558339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:26.606992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 500/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:26 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:26.631822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618f52d-a4b7-4a6f-9eb9-c3710cb199aa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:26.754737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618f52d-a4b7-4a6f-9eb9-c3710cb199aa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:26.799493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1add8df-cb67-4ee3-a58a-3568cd9f39d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:26.800613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1add8df-cb67-4ee3-a58a-3568cd9f39d9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:26.801120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1add8df-cb67-4ee3-a58a-3568cd9f39d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:26.801572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1add8df-cb67-4ee3-a58a-3568cd9f39d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:26.803447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1add8df-cb67-4ee3-a58a-3568cd9f39d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:26.805134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1add8df-cb67-4ee3-a58a-3568cd9f39d9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:26.807657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 520/1020] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:26.813302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:26.926587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8692-ddec-4988-818d-ef1118702a34 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:26.966215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f391a-e53e-415d-bb7c-1a1da764f43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:26.966846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f391a-e53e-415d-bb7c-1a1da764f43a None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:26.967134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f391a-e53e-415d-bb7c-1a1da764f43a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:26.967412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f391a-e53e-415d-bb7c-1a1da764f43a None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:26.968157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f391a-e53e-415d-bb7c-1a1da764f43a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:26.969625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f391a-e53e-415d-bb7c-1a1da764f43a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:26.995513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 501/1021] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:07:26 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:27.141854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa6efd-62ce-4f2d-95db-03be207dca4a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:27.142060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa6efd-62ce-4f2d-95db-03be207dca4a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:27.142363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa6efd-62ce-4f2d-95db-03be207dca4a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:27.144212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 521/1022] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:27.150216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4ca7f-f6b1-46e6-82b9-c5bfc75dc043 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:27.151068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4ca7f-f6b1-46e6-82b9-c5bfc75dc043 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:27.151338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4ca7f-f6b1-46e6-82b9-c5bfc75dc043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:27.173577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6be4ca7f-f6b1-46e6-82b9-c5bfc75dc043 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:27.174826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6be4ca7f-f6b1-46e6-82b9-c5bfc75dc043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:27.197342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:27.289923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 502/1023] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:27.316341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa6efd-62ce-4f2d-95db-03be207dca4a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:27.392409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:27.444145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa6efd-62ce-4f2d-95db-03be207dca4a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:27.486297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4a472-16d0-4c7c-961e-5833a9655576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:27.486978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4a472-16d0-4c7c-961e-5833a9655576 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:27.487434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4a472-16d0-4c7c-961e-5833a9655576 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:27.487948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4a472-16d0-4c7c-961e-5833a9655576 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:27.489217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4a472-16d0-4c7c-961e-5833a9655576 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:27.491782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4a472-16d0-4c7c-961e-5833a9655576 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:27.521394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 522/1024] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:27 2022] POST /identity/v3/services => generated 242 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:28.053535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e202-c124-4e7a-88ca-9134adc035a0 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:28.054114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e202-c124-4e7a-88ca-9134adc035a0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:28.054114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e202-c124-4e7a-88ca-9134adc035a0 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:28.056047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 503/1025] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:28.062040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:28.062329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:28.062814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:28.083753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:28.085214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:28.188065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 523/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:28.212197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e202-c124-4e7a-88ca-9134adc035a0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:28.335318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e202-c124-4e7a-88ca-9134adc035a0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:28.379826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:28.380746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:28.381186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:28.381650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:28.394920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:28.400390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:28.401508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:28.402340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:28.408250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 504/1027] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Aug 1 14:07:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:28.411150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:28.475162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:28.543624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80557697-4726-469d-9ac4-fb28fcd3be06 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:28.586722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:28.587339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:28.587712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:28.588644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:28.589617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:28.589939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:28.591682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:28.606706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 524/1028] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:07:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:28.608182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:28.625570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:28.748748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df834bd-9e86-49e3-bc0e-7800e15030f6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:28.788007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:28.788650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:28.788925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:28.789193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:28.800870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:28.802544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:28.819612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:28.820687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:28.822900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:28.822900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:28.822900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:28.822900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:28.824097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 505/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:28.827219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:28.966323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0425-eed9-47fd-86bc-ad5813ca36d9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:28.981687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:29.016084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.016997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.017437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.017866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.018722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:29.019176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:29.020741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b365d-78e4-4ef4-ad1f-8261e1bfd757 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:29.034947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 525/1030] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:29.041843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:29.135891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:29.193754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc83a-a492-4c31-9f5c-d5448f1e7648 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.194136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc83a-a492-4c31-9f5c-d5448f1e7648 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.194349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc83a-a492-4c31-9f5c-d5448f1e7648 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.196004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 526/1031] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:29.196232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4461f-162a-46b1-a016-f148531ab009 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:29.202021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.202426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.202662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.223430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:29.224619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:29.269650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.270733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.271483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.271794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.286596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:29.290047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:29.306198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 527/1032] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:29.313989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:29.315065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:07:29.316369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:29.319806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 506/1033] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:29.325608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:29.332624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:29.450506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9158f-fea6-4fb5-8c20-3d5eb539d092 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:29.461669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4db8a9-42aa-40c5-9447-905fea37c0d3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:29.464031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:29.481696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:29.494877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.495689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.495971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.496295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.508484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:29.511559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.512642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.513191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.513645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:29.514112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:29.515007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:29.521411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:29.522225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:29.522315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 507/1034] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:29.522397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:29.522397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:29.525117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:29.554121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 528/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:29.557055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:29.658975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f5062-f864-45bd-b59c-9dd88e24b6f2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:29.702609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea6ec0-c7bd-47cb-b120-a2deb6d51178 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:29.703659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.704296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.704579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.704856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.705592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:29.705902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:29.707287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db21cf1-4973-47ec-8ec5-7b77f0788df6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:29.717290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 508/1036] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:29.720813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a7ded-c2ac-4106-8d58-59466d81a16d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.721105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a7ded-c2ac-4106-8d58-59466d81a16d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.721417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a7ded-c2ac-4106-8d58-59466d81a16d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.722732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 509/1037] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:29.723864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a7ded-c2ac-4106-8d58-59466d81a16d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:29.743292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.743857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.744167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.744434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.776675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:29.778438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:29.827225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:29.828731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:29.839743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a7ded-c2ac-4106-8d58-59466d81a16d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:29.892860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64835ef3-82ac-46ac-aa43-a306a4d4ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:29.893520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64835ef3-82ac-46ac-aa43-a306a4d4ea9d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.893794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64835ef3-82ac-46ac-aa43-a306a4d4ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.894072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64835ef3-82ac-46ac-aa43-a306a4d4ea9d None admin] PATH_INFO: `/v3/users/c907d1ca361747959bb6fd9a462cf34a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.909384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64835ef3-82ac-46ac-aa43-a306a4d4ea9d None admin] RBAC: Authorizing `identity:get_user(user_id=c907d1ca361747959bb6fd9a462cf34a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:29.910953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64835ef3-82ac-46ac-aa43-a306a4d4ea9d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:29.912894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 510/1038] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/users/c907d1ca361747959bb6fd9a462cf34a => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:29.914117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:29.914921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:29.914921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:29.933255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:29.934476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:29.940485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 529/1039] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:07:29.941333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.008144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.012859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 511/1040] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:30.014056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.057543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.059336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70ac2-b651-4ed7-b1fe-b2661c654a37 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.098389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.099024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.099623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.099623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.140097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448db33-42a0-4a51-8cdd-664aa5f42fc1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.180789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.182041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.182161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.182352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.182656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.183726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.195670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.197360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:30.215275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:30.217512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:30.218601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:30.218601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:30.218601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:30.218601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:30.218601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 512/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:30.218601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.264601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.312703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 530/1042] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:30.313916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.328979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.346178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8a159-2533-413a-8602-06471505a531 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.389765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.390838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.391411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.391982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] PATH_INFO: `/v3/services/image` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.393225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.395906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:30.413052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:30.414436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:30.415405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 513/1043] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:30.416960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.451994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a85757-cbe2-4368-ac5d-3fe6648104d0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.499871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.500508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.500791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.501067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.501787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:30.502220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.503671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.519536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 531/1044] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:30.520329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.546861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.572781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceec17c-526c-4c8f-93b7-c4ebfa602c87 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.609519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.610185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.610523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.610868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.612148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c74d0611d0ae4e8780ad394926f3d8ed)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.613639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.615450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 514/1045] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:30.616275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.647955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.652021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd597e32-435f-401c-a342-f93f8a61ea45 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.695849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.696480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.696770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.697057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.697787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] RBAC: Adding query filter params (name=image) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:30.698097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.699557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.705528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 532/1046] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:30.706526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.766823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989ab50-722c-4431-b697-c12e582a29d5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.815233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aa98a-5594-485b-923d-fe449544046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.816445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aa98a-5594-485b-923d-fe449544046f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.817008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aa98a-5594-485b-923d-fe449544046f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.817345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aa98a-5594-485b-923d-fe449544046f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.825153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620aa98a-5594-485b-923d-fe449544046f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.826677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620aa98a-5594-485b-923d-fe449544046f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.828054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-620aa98a-5594-485b-923d-fe449544046f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:30.840078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:30.843689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 515/1047] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:30.845633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aa98a-5594-485b-923d-fe449544046f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.855303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8fdd-4063-43f9-ab9b-266a98e528ba None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:30.892462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:30.893399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:30.893680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:30.893960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:30.894753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed, user.id=c907d1ca361747959bb6fd9a462cf34a) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:30.895065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:30.896557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:30.918428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 533/1048] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/role_assignments?user.id=c907d1ca361747959bb6fd9a462cf34a&scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:30.918428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:30.961588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:31.006716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aa98a-5594-485b-923d-fe449544046f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:31.053067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:31.053729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:31.054011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:31.054262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:31.054946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] RBAC: Adding query filter params (type=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:31.055268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:31.057043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:31.058314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb773591-e8f2-4111-82d3-746a1256712f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:31.070012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 516/1049] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:31.070903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:31.111928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:31.113009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:31.113462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:31.113840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:31.114620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:31.114935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:31.116499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:31.133440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 534/1050] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:31.140407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:31.159111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:31.230878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f296a4-9460-4593-885f-989f299c0c98 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:31.283751 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:31.284758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:31.285591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:31.286549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:31.287868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] RBAC: Adding query filter params (interface=public, service_id=fcd43cd2bf654f8d852896e7c9011de4, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:31.288374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:31.291118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:31.293333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb409374-f884-4743-bffb-eb5f3090abeb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:31.300271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 517/1051] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:07:31 2022] GET /identity/v3/endpoints?service_id=fcd43cd2bf654f8d852896e7c9011de4&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:31.306716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:31.319202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:31.354761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:31.356067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:31.356556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:31.357011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:31.358203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] RBAC: Adding query filter params (role.id=3a2df3b6646746de9b17517a42ce6916, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=15b5f371cd614e3bb246b015818a97f2) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:31.358811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:31.360892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612994b7-b6e8-44c8-93cc-1e9bad33479c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:31.368426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 535/1052] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:31 2022] GET /identity/v3/role_assignments?user.id=15b5f371cd614e3bb246b015818a97f2&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=3a2df3b6646746de9b17517a42ce6916 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:31.488078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3752-fa3b-465c-86e0-3e2466cf756a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:31.549996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63494932-f84d-4be4-9f98-860850019ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:31.549996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63494932-f84d-4be4-9f98-860850019ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:31.550853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63494932-f84d-4be4-9f98-860850019ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:31.550853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63494932-f84d-4be4-9f98-860850019ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:31.551346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63494932-f84d-4be4-9f98-860850019ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:31.552658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63494932-f84d-4be4-9f98-860850019ae9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:31.561480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 518/1053] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:07:31 2022] GET /identity/v3/services => generated 1957 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:07:33.130071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960b410-7c6c-4c0a-8c1c-f465f63d1b8f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.130671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960b410-7c6c-4c0a-8c1c-f465f63d1b8f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.130671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960b410-7c6c-4c0a-8c1c-f465f63d1b8f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.133468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 536/1054] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:33.139519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.148123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.148123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.162070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:33.163306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:33.256265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:33.266085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 519/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:33.306214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960b410-7c6c-4c0a-8c1c-f465f63d1b8f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:33.403579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:33.439260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960b410-7c6c-4c0a-8c1c-f465f63d1b8f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:33.479597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02792f-6b11-4968-9d08-2ad2986d10af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:33.480381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02792f-6b11-4968-9d08-2ad2986d10af None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.480684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02792f-6b11-4968-9d08-2ad2986d10af None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.480994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02792f-6b11-4968-9d08-2ad2986d10af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.496167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02792f-6b11-4968-9d08-2ad2986d10af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bb02792f-6b11-4968-9d08-2ad2986d10af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:33.504830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:33.506055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:33.512127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 537/1056] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:07:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:33.514426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:33.521384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd6152-2d62-4b66-89e0-00fc8afd358d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.521384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd6152-2d62-4b66-89e0-00fc8afd358d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.521384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd6152-2d62-4b66-89e0-00fc8afd358d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.522926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 538/1057] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:33.527685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.528332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.528832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.548016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:33.549250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:33.629002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 539/1058] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:33.642538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcf4d1-12b3-49db-8521-fcc7ecbe7d63 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:33.653360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:33.697178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:33.697973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.698154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.698385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.699258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:33.699435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:33.701561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:33.713642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 520/1059] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:07:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:33.721045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:33.743626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:33.799346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0999537-916a-4f6c-8c29-a1e96b53be2e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:33.821903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:33.844333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888f89a-1e69-4dcb-b427-29e6c57eddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:33.845042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f89a-1e69-4dcb-b427-29e6c57eddcf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.845277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f89a-1e69-4dcb-b427-29e6c57eddcf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.845552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f89a-1e69-4dcb-b427-29e6c57eddcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.857905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888f89a-1e69-4dcb-b427-29e6c57eddcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6888f89a-1e69-4dcb-b427-29e6c57eddcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:33.862935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:33.864256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:33.865149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 540/1060] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Aug 1 14:07:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:33.865981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaed74-718d-4287-9f6f-ae8aeadd8600 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.866168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaed74-718d-4287-9f6f-ae8aeadd8600 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.866513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaed74-718d-4287-9f6f-ae8aeadd8600 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.868217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 541/1061] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:33.871037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbaed74-718d-4287-9f6f-ae8aeadd8600 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:33.881388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d4e66-6fc2-4174-a026-3c20216eb9dc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:33.924345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832d51b-9092-4c89-804a-57c136c13afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:33.924909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832d51b-9092-4c89-804a-57c136c13afc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.925194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832d51b-9092-4c89-804a-57c136c13afc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.925471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832d51b-9092-4c89-804a-57c136c13afc None admin] PATH_INFO: `/v3/users/c907d1ca361747959bb6fd9a462cf34a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.926670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832d51b-9092-4c89-804a-57c136c13afc None admin] RBAC: Authorizing `identity:get_user(user_id=c907d1ca361747959bb6fd9a462cf34a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:33.928148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832d51b-9092-4c89-804a-57c136c13afc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:33.929895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 521/1062] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:33 2022] GET /identity/v3/users/c907d1ca361747959bb6fd9a462cf34a => generated 285 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:33.931106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:33.931420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:33.932351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:33.958118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:33.959354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:34.015917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbaed74-718d-4287-9f6f-ae8aeadd8600 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.044392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.047994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 522/1063] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:34.048867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.061963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.062588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.062976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.063406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.064041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:34.064354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.065743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.076944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 542/1064] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:07:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:34.077927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.131319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.186843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff028f-0247-46d9-951d-cc5931ecd7ff None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.199623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92bc9-8858-4843-ace7-614a1a79614d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.225528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.226181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.226463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.226740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.227974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] RBAC: Authorizing `identity:get_project(project_id=c74d0611d0ae4e8780ad394926f3d8ed)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.229953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.231684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 523/1065] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:34.232455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.245940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.246591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.246880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.247146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] PATH_INFO: `/v3/services/image` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.247806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.249203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:34.260014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:34.261214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:34.262052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 543/1066] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:34.263018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.328701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.349003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175d933-fcc8-4d69-82e1-b8964a9d1229 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.382930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.391227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c23073-2ca5-4497-a6ad-209d62bbdb0d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.393228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.394275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.394739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.395207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.406880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.408505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.427436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:34.427862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:34.428841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:34.429720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:34.429720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:34.429720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:34.429720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:34.429948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.429948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.429948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed/users/c907d1ca361747959bb6fd9a462cf34a/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.430127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 524/1067] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:34.430127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.436132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] RBAC: Authorizing `identity:create_grant(project_id=c74d0611d0ae4e8780ad394926f3d8ed, user_id=c907d1ca361747959bb6fd9a462cf34a, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.437689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.457582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 544/1068] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:07:34 2022] PUT /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed/users/c907d1ca361747959bb6fd9a462cf34a/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:07:34.458810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.582748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ad62-c861-47cb-9aea-f86a561a8790 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.615089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a009936-c35d-4d5b-a6a6-82f90a8ed650 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.633445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.637629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.638276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.638582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.638921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.639768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] RBAC: Adding query filter params (name=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:34.640083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.642061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.649297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 525/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:34.659500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.673352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.673977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.674258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.674531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.675230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:34.675565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.677639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.699648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 545/1070] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:34.701502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.724388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.797915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaece19-a83f-45e2-abf3-37b967cffb9c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.823595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5119270-4ef3-4421-a49d-acb98be86ae9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:34.845335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.845977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.846266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.846543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.847249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] RBAC: Adding query filter params (type=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:34.847591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.849340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.859516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 526/1071] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:34.861141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.863666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:34.864520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:34.865690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:34.866127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:34.878216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:34.879851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:34.893294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:34.894474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:34.895532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 546/1072] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:34.902969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:34.946865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:34.997027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714293e-4593-4e97-b201-7ea5b9b39063 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:35.010924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:35.037694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b864ed0-f244-4f5f-8d56-2814b0aad466 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:35.046761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:35.047743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:35.048028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:35.048425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:35.049146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:35.050827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:35.082292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:35.083102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:35.083381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:35.083695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:35.091734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 527/1073] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:07:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:35.091945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:35.091945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:35.091945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:35.110125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 547/1074] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:07:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:35.116574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:35.266648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d0492-e55d-4b02-81ce-336004d06d67 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:35.318630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:35.319287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:35.319597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:35.319873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:35.332523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:35.333985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:35.352676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:35.354214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:35.355202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:35.355202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:35.355202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:35.355202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:35.355202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 528/1075] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:35.360282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:35.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:35.493060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2d08d-3ef8-418b-b61a-156c2667d7ab None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:35.533107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:35.534184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:35.534608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:35.535035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:35.535944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:35.536411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:35.537933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:35.553832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 548/1076] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:35.559610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:35.577224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:35.704751 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d9fe-017d-409b-a7b5-eb4406964304 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:35.755208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:35.755972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:35.756254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:35.756519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:35.764786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:35.766656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:35.787068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:35.788169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:35.789049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 529/1077] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:35.796456 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:35.937906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3e868-a95d-443b-9e83-9f93ff360bf0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:35.981114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:35.981753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:35.982030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:35.982306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:35.983075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:35.983384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:35.984990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0f3a2-d138-489f-8e0f-688ea2e96e94 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:36.004985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 549/1078] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:36.007449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:36.093908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:36.137061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569cf6c-202e-475c-a5f5-0786d7ff0465 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:36.177396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50c40f-96ef-4d3b-a558-5c8795c20bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:36.178334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50c40f-96ef-4d3b-a558-5c8795c20bec None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:36.178801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50c40f-96ef-4d3b-a558-5c8795c20bec None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:36.179224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50c40f-96ef-4d3b-a558-5c8795c20bec None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/15b5f371cd614e3bb246b015818a97f2/roles/3a2df3b6646746de9b17517a42ce6916` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:36.182933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50c40f-96ef-4d3b-a558-5c8795c20bec None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=15b5f371cd614e3bb246b015818a97f2, role_id=3a2df3b6646746de9b17517a42ce6916)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:36.184722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50c40f-96ef-4d3b-a558-5c8795c20bec None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:36.207524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 530/1079] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:07:36 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/15b5f371cd614e3bb246b015818a97f2/roles/3a2df3b6646746de9b17517a42ce6916 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:07:36.587288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2378017-6125-48a0-9c0a-b9a99103b88d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:36.588716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2378017-6125-48a0-9c0a-b9a99103b88d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:36.588716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2378017-6125-48a0-9c0a-b9a99103b88d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:36.589953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 550/1080] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:36.595558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:36.596051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:36.596128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:36.617988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:36.620004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:36.719498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 531/1081] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:36.747152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2378017-6125-48a0-9c0a-b9a99103b88d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:36.757369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:36.883788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2378017-6125-48a0-9c0a-b9a99103b88d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:36.937480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:36.938131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:36.938410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:36.938733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:36.956175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:36.963363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:36.964745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:36.966309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 551/1082] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:36.972794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.045527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:37.093199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6a614-7ceb-41a4-be0f-01044a8bc9c3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:37.133317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:37.134083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.134965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.135255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.136070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:37.136384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:37.138004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84197-a278-499d-b1c5-80e32d0fed3f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:37.149015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 532/1083] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:37.154699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.234891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:37.236832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636d97-7d31-4c1b-a63a-4ac851377588 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.237335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636d97-7d31-4c1b-a63a-4ac851377588 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.237800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636d97-7d31-4c1b-a63a-4ac851377588 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.239443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 533/1084] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:37.245859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.246157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.246439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.265909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:37.267123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:37.287386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd16149-d1e4-4716-89f5-0189293a62bc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:37.332531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:37.333251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.333528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.333799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] PATH_INFO: `/v3/users/c907d1ca361747959bb6fd9a462cf34a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.335004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] RBAC: Authorizing `identity:get_user(user_id=c907d1ca361747959bb6fd9a462cf34a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:37.336486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:37.339512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 552/1085] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:37 2022] GET /identity/v3/users/c907d1ca361747959bb6fd9a462cf34a => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:37.344532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.363509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 534/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:37.388064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.465313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:37.476829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911a0a-4b8b-4c0e-8a05-e8adc5f91e3f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:37.522746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:37.523373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.523710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.523990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.527342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:37.531339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bdd58-d860-449c-a405-8913d4983d6f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:37.580344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:37.580963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.581244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.581516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.619112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:37.620651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:37.670861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:37.673660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:37.744301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 553/1087] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:37.750398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.751411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:37.789719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:37.813508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 535/1088] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:07:37.814814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.877820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67df8e-c3d1-4d4b-94f6-412fd2f92f0f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:37.919655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:37.920601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:37.921086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:37.921545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:37.925519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] RBAC: Authorizing `identity:get_project(project_id=c74d0611d0ae4e8780ad394926f3d8ed)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:37.926987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:37.929359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 554/1089] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:37 2022] GET /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:37.935449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:37.964894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec0b8-081e-45df-9d8c-c4bc01380f9d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:38.008762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:38.009951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.010239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.010517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.025562 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:38.027217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:38.049582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:38.050580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:38.051519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:38.051519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:38.051519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:38.051519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:38.052954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 536/1090] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:38.059595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:38.069436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:38.078926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecec023-3fc4-4cc2-979e-02bbb7789491 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:38.080738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:38.122020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:38.122766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.123047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.123324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.124175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed, user.id=c907d1ca361747959bb6fd9a462cf34a) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:38.124500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:38.125926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624718c1-e7f9-443f-b6c6-3345b2eebdbd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:38.139565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 555/1091] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:37 2022] GET /identity/v3/role_assignments?user.id=c907d1ca361747959bb6fd9a462cf34a&scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:38.208203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c7860-a0c5-4366-a684-4f410b99f15e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:38.263359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:38.264059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.264340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.264620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.265365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:38.265674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:38.267612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d93bbf-58dc-461c-81a9-330a725fcc76 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:38.291514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 537/1092] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:38.372349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ff411-f943-40c0-a2f2-5d93118a4c87 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.372687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ff411-f943-40c0-a2f2-5d93118a4c87 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.372915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ff411-f943-40c0-a2f2-5d93118a4c87 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.374645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 556/1093] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:38.380551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.380881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.381121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.402351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:38.403721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:38.483344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:38.492954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 538/1094] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:38.516968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ff411-f943-40c0-a2f2-5d93118a4c87 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:38.653481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ff411-f943-40c0-a2f2-5d93118a4c87 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:38.694383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:38.695036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.695316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.695626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.708628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:38.713693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:38.714620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:38.715709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 557/1095] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Aug 1 14:07:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:38.721434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:38.828574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810641e6-e4b5-4a26-9fb8-950047c097b6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:38.870071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:38.870694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:38.871075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:38.871360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:38.872125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:38.872434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:38.873825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:38.883157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 539/1096] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:07:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:38.889025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:38.920280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:39.002999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049b271-ab92-44f5-841c-48c023caa2d1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:39.041706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:39.042355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:39.042630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:39.042900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:39.059368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:39.059368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:39.075302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:39.076661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:39.077657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:39.077657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:39.077657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:39.077657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:39.077657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 558/1097] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:07:39.082925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:39.146588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:39.191712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8aac5-b757-4f93-864c-c189f91df3b8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:39.236668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:39.237290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:39.237595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:39.237869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:39.238665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:39.238970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:39.240428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:39.253797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 540/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:39.260909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:39.348459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:39.375624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613deb80-0b88-4a3e-93d2-d1e649fda48e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:39.410511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:39.411168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:39.411474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:39.411757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:39.420699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:39.422167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:07:39.434181 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:39.435116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:39.436286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 559/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Aug 1 14:07:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:39.441471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:39.550697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad495a2-5c46-488b-81a3-79a811af312f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:39.568917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:39.588308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:39.588943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:39.589231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:39.589510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:39.590352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:39.590664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:39.592135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:39.612979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 541/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Aug 1 14:07:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:39.618838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:39.759919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef705-be45-4cfd-b071-6598747e5e76 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:39.796584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:39.797206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:39.797482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:39.797748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:39.810975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:39.880730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:39.882110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.010723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 560/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:40.016791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.146600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b9500-ff4c-4775-a934-8b2423ba7b13 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.185188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.185848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.186130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.186408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.198390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.199898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f36c7d09-0271-4eec-874f-9da2f0de9065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:07:40.219836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:40.220806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:40.221872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:07:40.221872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:07:40.221872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:07:40.221872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:07:40.221872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 542/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:40.227444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.270610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:40.287955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb1dd-44ed-4f08-982d-b180c73f8b51 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.288242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb1dd-44ed-4f08-982d-b180c73f8b51 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.288523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb1dd-44ed-4f08-982d-b180c73f8b51 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.289881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 543/1103] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:07:40.296013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.296321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.296670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.317671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:40.318900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:40.341797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20cb32-d102-4653-8674-6a6103f73422 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.345877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:40.381374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.381999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.382276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.382549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.383240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:40.383583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.385203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.397295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 561/1104] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:40.398855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 544/1105] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:07:40.402918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.423422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.513018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:40.517733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f728-4da7-4592-8351-19f4351fc0f8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.533284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9d40a-2c9d-466b-abf1-c3aabca62117 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.557732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.558376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.558659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.558941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.566748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.568258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.569884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.570530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.570809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.571087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.572414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.573770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.575711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 545/1106] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:40.581651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:07:40.582280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:40.583354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:40.584325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 562/1107] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:40.590657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.612565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:40.689749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8537af-c7ce-479d-ba44-2c4bad2522c3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.722628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6319cc-7660-4833-8902-383b35d78e1f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.728537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.729177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.729461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.729737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.730437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.732101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.755491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 546/1108] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:07:40 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:07:40.771737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.772392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.772675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.772950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.773770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:40.774081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.776168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7072-2286-4661-b31d-f5b83e69c6a0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.792154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 563/1109] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:40.799588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:40.901166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:40.923415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08249ec-0bb5-4a49-b6d6-5aee7991d5b0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:40.973636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:40.974363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:40.974604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:40.974802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:40.975640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] RBAC: Adding query filter params (role.id=3a2df3b6646746de9b17517a42ce6916, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=15b5f371cd614e3bb246b015818a97f2) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:40.976042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:40.977434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca93708-7977-4113-aa4b-c80469a5ba04 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:40.985322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 547/1110] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:40 2022] GET /identity/v3/role_assignments?user.id=15b5f371cd614e3bb246b015818a97f2&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=3a2df3b6646746de9b17517a42ce6916 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:42.914074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a2ae-006f-4f30-ae40-9197d56e04fa None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:42.914629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a2ae-006f-4f30-ae40-9197d56e04fa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:42.914629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a2ae-006f-4f30-ae40-9197d56e04fa None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:42.916648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 564/1111] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:42.922663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:42.923224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:42.923224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:42.942524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:42.943990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:43.014598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 548/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:43.038420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a2ae-006f-4f30-ae40-9197d56e04fa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:43.059343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:43.155355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a2ae-006f-4f30-ae40-9197d56e04fa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:43.192308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:43.192857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:43.193138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:43.193410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] PATH_INFO: `/v3/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:43.194873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] RBAC: Authorizing `identity:get_role(role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:43.196291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:43.197718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 565/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:43 2022] GET /identity/v3/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:43.206986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:43.290113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:43.315201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ac36b-9500-4fc7-bc36-1860c0990d60 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:43.352115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:43.352770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:43.353056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:43.353334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] PATH_INFO: `/v3/users/62bdbc68e6094fa39a2fcea944f7f0ce` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:43.368900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] RBAC: Authorizing `identity:get_user(user_id=62bdbc68e6094fa39a2fcea944f7f0ce)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:43.370388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:43.372167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 549/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:43 2022] GET /identity/v3/users/62bdbc68e6094fa39a2fcea944f7f0ce => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:43.378080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:43.468478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:43.484680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565e0a-5f68-424c-a5f5-84799ea92a48 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:43.525336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:43.525980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:43.526259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:43.526531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:43.596797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:43.598520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:43.690135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:43.707418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 566/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:43.715414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:43.829463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad110b80-7405-4b8e-a64f-46f0861660f4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:43.865695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588f9c2-1ffa-4a9f-9427-377f5ecae906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:43.866343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588f9c2-1ffa-4a9f-9427-377f5ecae906 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:43.866624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588f9c2-1ffa-4a9f-9427-377f5ecae906 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:43.866902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588f9c2-1ffa-4a9f-9427-377f5ecae906 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:43.868096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588f9c2-1ffa-4a9f-9427-377f5ecae906 None admin] RBAC: Authorizing `identity:get_project(project_id=c74d0611d0ae4e8780ad394926f3d8ed)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:43.870105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588f9c2-1ffa-4a9f-9427-377f5ecae906 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:43.871833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 550/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:43 2022] GET /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:43.878078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:43.991744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2511ebb-7007-4558-8b96-9d8e51299870 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:44.027094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947661b4-9926-46da-810a-229b44c44085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:44.027746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947661b4-9926-46da-810a-229b44c44085 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:44.028033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947661b4-9926-46da-810a-229b44c44085 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:44.028311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947661b4-9926-46da-810a-229b44c44085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:44.029101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947661b4-9926-46da-810a-229b44c44085 None admin] RBAC: Adding query filter params (role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed, user.id=62bdbc68e6094fa39a2fcea944f7f0ce) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:44.029419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947661b4-9926-46da-810a-229b44c44085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:44.030871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947661b4-9926-46da-810a-229b44c44085 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:44.036786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 567/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:43 2022] GET /identity/v3/role_assignments?user.id=62bdbc68e6094fa39a2fcea944f7f0ce&scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:46.129222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc2fc-9ee3-4604-9264-d4c14d82e833 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:46.129914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc2fc-9ee3-4604-9264-d4c14d82e833 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:46.129914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc2fc-9ee3-4604-9264-d4c14d82e833 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:46.131722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 551/1118] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:46.138739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:46.139321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:46.139321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:46.159047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:46.160298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:46.213977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:46.237439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 568/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:46.260841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc2fc-9ee3-4604-9264-d4c14d82e833 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:46.263879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:46.373870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc2fc-9ee3-4604-9264-d4c14d82e833 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:46.414608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:46.415202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:46.415546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:46.415807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] PATH_INFO: `/v3/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:46.417373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] RBAC: Authorizing `identity:get_role(role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:46.418823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:46.420276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 552/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:46 2022] GET /identity/v3/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:46.426403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:46.548488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa74c7d-ab6b-4645-8066-fba713fdd662 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:46.591207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:46.591905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:46.592323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:46.592454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] PATH_INFO: `/v3/users/62bdbc68e6094fa39a2fcea944f7f0ce` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:46.593779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=62bdbc68e6094fa39a2fcea944f7f0ce)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:46.595260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:46.597162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 569/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:46 2022] GET /identity/v3/users/62bdbc68e6094fa39a2fcea944f7f0ce => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:46.603648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:46.684593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:46.723210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152fdab-e780-4fc0-a75c-1207330576a6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:46.764945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ca656-1c02-4f7a-820d-95eff09e5d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:46.765634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ca656-1c02-4f7a-820d-95eff09e5d61 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:46.765927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ca656-1c02-4f7a-820d-95eff09e5d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:46.766202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ca656-1c02-4f7a-820d-95eff09e5d61 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:46.767405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ca656-1c02-4f7a-820d-95eff09e5d61 None admin] RBAC: Authorizing `identity:get_project(project_id=c74d0611d0ae4e8780ad394926f3d8ed)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:46.768885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ca656-1c02-4f7a-820d-95eff09e5d61 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:46.770603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 553/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:46 2022] GET /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:07:46.777219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:46.831482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:46.892640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c9fae-3a4c-45f7-abe3-ef76f2459ed2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:46.935499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fd278-2370-4480-bf6a-ab0185ff6428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:46.936126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd278-2370-4480-bf6a-ab0185ff6428 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:46.936409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd278-2370-4480-bf6a-ab0185ff6428 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:46.936690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd278-2370-4480-bf6a-ab0185ff6428 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed/users/62bdbc68e6094fa39a2fcea944f7f0ce/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:46.939175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fd278-2370-4480-bf6a-ab0185ff6428 None admin] RBAC: Authorizing `identity:create_grant(project_id=c74d0611d0ae4e8780ad394926f3d8ed, user_id=62bdbc68e6094fa39a2fcea944f7f0ce, role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:46.940687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fd278-2370-4480-bf6a-ab0185ff6428 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:46.957164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 570/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:07:46 2022] PUT /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed/users/62bdbc68e6094fa39a2fcea944f7f0ce/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:07:49.050506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af7d4f-004a-4726-9e30-45cf8673e8ca None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:49.051083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af7d4f-004a-4726-9e30-45cf8673e8ca None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:49.051083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af7d4f-004a-4726-9e30-45cf8673e8ca None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:49.052460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 554/1124] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:49.057895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:49.058373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:49.058481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:49.076616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:49.077827 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:49.146188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 571/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:49.170690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af7d4f-004a-4726-9e30-45cf8673e8ca None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:49.271627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af7d4f-004a-4726-9e30-45cf8673e8ca None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:49.306796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:49.307567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:49.307993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:49.308131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] PATH_INFO: `/v3/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:49.310809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] RBAC: Authorizing `identity:get_role(role_id=5318f33e1a0f4fb6831bc2a3de8dc4dd)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:49.312215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:49.313590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 555/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:49 2022] GET /identity/v3/roles/5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:49.319727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:49.361421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:49.422712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a03a86-a531-4a8d-9ba7-fc49e6de195f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:49.457643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:49.458280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:49.458557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:49.458836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] PATH_INFO: `/v3/users/62bdbc68e6094fa39a2fcea944f7f0ce` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:49.460027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] RBAC: Authorizing `identity:get_user(user_id=62bdbc68e6094fa39a2fcea944f7f0ce)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:49.461465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:49.463749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 572/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:49 2022] GET /identity/v3/users/62bdbc68e6094fa39a2fcea944f7f0ce => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:49.469655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:49.479564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:49.584361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e820e2-cda0-4f78-8f4c-2cf790591076 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:49.618908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:49.619549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:49.619834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:49.620108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:49.690148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:49.691653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:49.701007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:49.795596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 556/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:49.801480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:49.902287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1228-d865-40b5-86d2-e1eb1499173e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:49.909378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:49.936388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc7bb6-b770-44c4-bbf3-015c04217a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:49.937031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc7bb6-b770-44c4-bbf3-015c04217a49 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:49.937315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc7bb6-b770-44c4-bbf3-015c04217a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:49.937593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc7bb6-b770-44c4-bbf3-015c04217a49 None admin] PATH_INFO: `/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:49.938749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc7bb6-b770-44c4-bbf3-015c04217a49 None admin] RBAC: Authorizing `identity:get_project(project_id=c74d0611d0ae4e8780ad394926f3d8ed)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:49.940207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc7bb6-b770-44c4-bbf3-015c04217a49 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:49.941886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 573/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:49 2022] GET /identity/v3/projects/c74d0611d0ae4e8780ad394926f3d8ed => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:49.948591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:50.049008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88795a8-047e-4f77-be64-a15c5dd6696d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:50.057292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:50.084843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:50.085457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:50.085739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:50.086012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:50.086796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] RBAC: Adding query filter params (role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd, scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed, user.id=62bdbc68e6094fa39a2fcea944f7f0ce) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:50.087108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:50.088611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0364e7-a807-4192-b0fe-7c685f2c879b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:50.094376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 557/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:49 2022] GET /identity/v3/role_assignments?user.id=62bdbc68e6094fa39a2fcea944f7f0ce&scope.project.id=c74d0611d0ae4e8780ad394926f3d8ed&role.id=5318f33e1a0f4fb6831bc2a3de8dc4dd => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:52.192604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a22bc0-46ba-40b0-8360-f49c49d455f9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:52.193147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a22bc0-46ba-40b0-8360-f49c49d455f9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:52.193147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a22bc0-46ba-40b0-8360-f49c49d455f9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:52.194513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 574/1131] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:52.200262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:52.200742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:52.200881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:52.222148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:52.224093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:52.295152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 558/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:52.318031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a22bc0-46ba-40b0-8360-f49c49d455f9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:52.425877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a22bc0-46ba-40b0-8360-f49c49d455f9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:52.462821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a037-9dc1-4635-8b5b-9706e7165237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:52.463508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a037-9dc1-4635-8b5b-9706e7165237 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:52.463865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a037-9dc1-4635-8b5b-9706e7165237 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:52.464143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a037-9dc1-4635-8b5b-9706e7165237 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:52.465323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a037-9dc1-4635-8b5b-9706e7165237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:52.466761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a037-9dc1-4635-8b5b-9706e7165237 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:52.468407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 575/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:52.475087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:52.579818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:52.583024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2689f-61d4-4bb6-b91b-376465ffc466 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:52.624946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0a371-7210-4bf7-9ab5-e6d4ceccf97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:52.625572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a371-7210-4bf7-9ab5-e6d4ceccf97f None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:52.625854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a371-7210-4bf7-9ab5-e6d4ceccf97f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:52.626132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a371-7210-4bf7-9ab5-e6d4ceccf97f None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:52.626837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0a371-7210-4bf7-9ab5-e6d4ceccf97f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:52.628331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0a371-7210-4bf7-9ab5-e6d4ceccf97f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:52.658089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 559/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:07:52 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:54.771203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1441b5-b58a-4f64-a170-9ab0eec54116 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:54.771811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1441b5-b58a-4f64-a170-9ab0eec54116 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:54.771811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1441b5-b58a-4f64-a170-9ab0eec54116 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:54.773149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 576/1135] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:54.778728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:54.779224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:54.779316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:54.798903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:54.800141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:54.871471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 560/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:54.896074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1441b5-b58a-4f64-a170-9ab0eec54116 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:54.956395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:55.017008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1441b5-b58a-4f64-a170-9ab0eec54116 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:55.054005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72108c96-028f-499f-9e18-ca69c98f07a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:55.054679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72108c96-028f-499f-9e18-ca69c98f07a6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:55.054960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72108c96-028f-499f-9e18-ca69c98f07a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:55.055233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72108c96-028f-499f-9e18-ca69c98f07a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:55.056489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72108c96-028f-499f-9e18-ca69c98f07a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:55.057927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72108c96-028f-499f-9e18-ca69c98f07a6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:55.059610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 577/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:07:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:55.065858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:55.160051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:55.181376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdde893-0eb5-4f18-a6ef-773cec7307b4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:55.224843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe50803-e1b6-422c-a4ee-8b0d65ce89b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:55.224843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe50803-e1b6-422c-a4ee-8b0d65ce89b9 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:55.225207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe50803-e1b6-422c-a4ee-8b0d65ce89b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:55.225371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe50803-e1b6-422c-a4ee-8b0d65ce89b9 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:55.226080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe50803-e1b6-422c-a4ee-8b0d65ce89b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:55.228142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe50803-e1b6-422c-a4ee-8b0d65ce89b9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:55.250769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 561/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:07:55 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:07:57.328810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8205a9-ef14-4f3e-9dba-bbd6db5957f8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:57.329408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8205a9-ef14-4f3e-9dba-bbd6db5957f8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:57.329408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8205a9-ef14-4f3e-9dba-bbd6db5957f8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:57.330728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 578/1139] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:57.336492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:57.337234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:57.337234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:57.364404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:57.366460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:57.460011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 562/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:07:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:07:57.484223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8205a9-ef14-4f3e-9dba-bbd6db5957f8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:57.595665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8205a9-ef14-4f3e-9dba-bbd6db5957f8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:57.607977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:57.636490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:57.637130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:57.637411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:57.637686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:57.653403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:07:57.658592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:07:57.659858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:07:57.661504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 579/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:07:57.665794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:57.733377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:57.768869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae964f-8a52-4a43-884f-866e335edfd5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:57.804065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:57.804691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:57.804975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:57.805252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:57.806012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:57.806334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:57.807728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:57.816255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 563/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:57.822662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:57.928985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d53e-9ac0-490c-9ce4-6d403282a6a2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:57.963389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:57.964062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:57.964336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:57.964609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] PATH_INFO: `/v3/users/db6570fea19646b79042de6ee0cd553d` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:57.978596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] RBAC: Authorizing `identity:get_user(user_id=db6570fea19646b79042de6ee0cd553d)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:57.980074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:57.981772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 580/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:07:57 2022] GET /identity/v3/users/db6570fea19646b79042de6ee0cd553d => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:57.987662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:58.087845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b1213-b214-493b-8288-088af4f71339 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:58.120122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:58.126352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:58.126976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:58.127311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:58.128129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:58.198829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:58.200257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:58.322729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 564/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:07:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:07:58.328884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:58.360305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:58.439771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019ac92-9569-4628-b24a-178fb41207dc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:58.480857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cc352-d3d5-46a0-9d49-7041a5203003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:58.481769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc352-d3d5-46a0-9d49-7041a5203003 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:58.482046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc352-d3d5-46a0-9d49-7041a5203003 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:58.482322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc352-d3d5-46a0-9d49-7041a5203003 None admin] PATH_INFO: `/v3/projects/89d358507e1949ec8089e48abc0dfde1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:58.483580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cc352-d3d5-46a0-9d49-7041a5203003 None admin] RBAC: Authorizing `identity:get_project(project_id=89d358507e1949ec8089e48abc0dfde1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:58.485034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cc352-d3d5-46a0-9d49-7041a5203003 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:58.486737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 581/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:07:58 2022] GET /identity/v3/projects/89d358507e1949ec8089e48abc0dfde1 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:07:58.493814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:58.530005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:58.607280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35434d-950f-4533-a970-98c064c69262 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:58.641443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:58.642067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:58.642347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:58.642622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:58.643414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=89d358507e1949ec8089e48abc0dfde1, user.id=db6570fea19646b79042de6ee0cd553d) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:07:58.643754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:58.645197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4942dc-d7fd-4787-8351-d50874cc5c9e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:58.652128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 565/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:07:58 2022] GET /identity/v3/role_assignments?user.id=db6570fea19646b79042de6ee0cd553d&scope.project.id=89d358507e1949ec8089e48abc0dfde1&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:00.657227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cafe9-7358-4811-b791-52f9e7b9f386 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:00.657780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cafe9-7358-4811-b791-52f9e7b9f386 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:00.657780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cafe9-7358-4811-b791-52f9e7b9f386 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:00.659301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 582/1147] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:00.664395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:00.664802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:00.664977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:00.682755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:00.684019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:00.751548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 566/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:00.774605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cafe9-7358-4811-b791-52f9e7b9f386 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:00.868505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:00.875510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cafe9-7358-4811-b791-52f9e7b9f386 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:00.908751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:00.909311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:00.909587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:00.909856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:00.919978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:08:00.924821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:08:00.925761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:08:00.926955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 583/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:08:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:08:00.932592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:00.957794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:01.032024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ba78a-e370-4613-a1de-fdb8abdcc56f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:01.068010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:01.068664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:01.068945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:01.069221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:01.069983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:01.070298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:01.072398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:01.083204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 567/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:08:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:01.090091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:01.207919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802e5b3-cfb4-4aba-821b-82efac2d1ff9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:01.244119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:01.244772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:01.245057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:01.245335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] PATH_INFO: `/v3/users/db6570fea19646b79042de6ee0cd553d` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:01.246549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] RBAC: Authorizing `identity:get_user(user_id=db6570fea19646b79042de6ee0cd553d)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:01.248024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:01.249663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 584/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:08:01 2022] GET /identity/v3/users/db6570fea19646b79042de6ee0cd553d => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:01.255616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:01.345047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:01.361705 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995d120-656b-4046-abaa-a9224c48498b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:01.396513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7150e6-27a2-47a8-ba0c-a1e15d8f556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:01.397162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7150e6-27a2-47a8-ba0c-a1e15d8f556a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:01.397445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7150e6-27a2-47a8-ba0c-a1e15d8f556a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:01.397724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7150e6-27a2-47a8-ba0c-a1e15d8f556a None admin] PATH_INFO: `/v3/projects/89d358507e1949ec8089e48abc0dfde1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:01.399115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7150e6-27a2-47a8-ba0c-a1e15d8f556a None admin] RBAC: Authorizing `identity:get_project(project_id=89d358507e1949ec8089e48abc0dfde1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:01.400596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7150e6-27a2-47a8-ba0c-a1e15d8f556a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:01.402291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 568/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:08:01 2022] GET /identity/v3/projects/89d358507e1949ec8089e48abc0dfde1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:01.408441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:01.434596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:01.512645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121523a-3ad5-471f-b977-fa58b29dd0b9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:01.548821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177762fe-e63f-4281-9ab6-67c167189336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:01.549492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177762fe-e63f-4281-9ab6-67c167189336 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:01.549774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177762fe-e63f-4281-9ab6-67c167189336 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:01.550053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177762fe-e63f-4281-9ab6-67c167189336 None admin] PATH_INFO: `/v3/projects/89d358507e1949ec8089e48abc0dfde1/users/db6570fea19646b79042de6ee0cd553d/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:01.552059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177762fe-e63f-4281-9ab6-67c167189336 None admin] RBAC: Authorizing `identity:create_grant(project_id=89d358507e1949ec8089e48abc0dfde1, user_id=db6570fea19646b79042de6ee0cd553d, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:01.553491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177762fe-e63f-4281-9ab6-67c167189336 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:01.568537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 585/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:08:01 2022] PUT /identity/v3/projects/89d358507e1949ec8089e48abc0dfde1/users/db6570fea19646b79042de6ee0cd553d/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:03.585021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1a4a-3620-4cb7-88ff-bd12cf170539 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:03.585674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1a4a-3620-4cb7-88ff-bd12cf170539 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:03.585674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1a4a-3620-4cb7-88ff-bd12cf170539 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:03.587611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 569/1154] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:03.593626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:03.594183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:03.594435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:03.613627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:03.614854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:03.692281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 586/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:03.716641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1a4a-3620-4cb7-88ff-bd12cf170539 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:03.820414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1a4a-3620-4cb7-88ff-bd12cf170539 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:03.856648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:03.857197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:03.857482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:03.857755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:03.868668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:08:03.873638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:08:03.874843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:08:03.875930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 570/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:08:03.881076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:03.897374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:03.983444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83d043-9592-4c09-860e-72b9f18c3fce None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:04.017424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:04.018032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:04.018307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:04.018575 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:04.019286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:04.019619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:04.020983 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:04.030081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 587/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:08:04.036264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:04.053631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:04.149741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca98d0c-0763-4db3-ab03-da175bf77405 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:04.190798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:04.191437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:04.191745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:04.192018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] PATH_INFO: `/v3/users/db6570fea19646b79042de6ee0cd553d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:04.193202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=db6570fea19646b79042de6ee0cd553d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:04.194649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:04.196928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 571/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:08:04 2022] GET /identity/v3/users/db6570fea19646b79042de6ee0cd553d => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:04.202915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:04.286215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:04.315549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204d8db-afba-4db3-b9a6-3b9fa7369c3a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:04.349839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:04.350462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:04.350748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:04.351089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:04.420996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:04.422468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:04.502250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:04.527425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 588/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:08:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:08:04.533533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:04.622971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:04.642132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1908be-ee0d-4446-82e0-b712094e5cf7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:04.683990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f9d59-cc37-4d96-8d79-586a998a0859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:04.684553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f9d59-cc37-4d96-8d79-586a998a0859 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:04.684834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f9d59-cc37-4d96-8d79-586a998a0859 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:04.685110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f9d59-cc37-4d96-8d79-586a998a0859 None admin] PATH_INFO: `/v3/projects/89d358507e1949ec8089e48abc0dfde1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:04.686984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f9d59-cc37-4d96-8d79-586a998a0859 None admin] RBAC: Authorizing `identity:get_project(project_id=89d358507e1949ec8089e48abc0dfde1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:04.689066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f9d59-cc37-4d96-8d79-586a998a0859 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:04.690821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 572/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:08:04 2022] GET /identity/v3/projects/89d358507e1949ec8089e48abc0dfde1 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:04.697925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:04.804080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081b9c5-c415-47c7-a0b2-336625682c07 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:04.840326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:04.840946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:04.841232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:04.841504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:04.842280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=89d358507e1949ec8089e48abc0dfde1, user.id=db6570fea19646b79042de6ee0cd553d) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:04.842593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:04.844172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53d9c3-fdd5-4c8b-9f1e-d40515cf0add None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:04.850278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 589/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:08:04 2022] GET /identity/v3/role_assignments?user.id=db6570fea19646b79042de6ee0cd553d&scope.project.id=89d358507e1949ec8089e48abc0dfde1&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:06.907779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333b51e-9af6-45e8-a732-866b38b43b4b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:06.908309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333b51e-9af6-45e8-a732-866b38b43b4b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:06.908309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333b51e-9af6-45e8-a732-866b38b43b4b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:06.909875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 573/1162] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:06.915714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:06.916219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:06.916358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:06.935363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:06.936574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:06.971286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:07.010618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 590/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:07.032417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333b51e-9af6-45e8-a732-866b38b43b4b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:07.139955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333b51e-9af6-45e8-a732-866b38b43b4b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:07.173791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:07.174324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:07.174599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:07.174868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:07.194557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:07.254828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:07.256260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:07.365716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 574/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:08:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:08:07.372619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:07.481161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106cfca-3069-43d4-9d93-59723f197f45 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:07.516897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:07.517547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:07.517829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:07.518111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:07.525489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:07.526964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-be86d8c1-69db-49ae-8005-ca3d205f1f7e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:08:07.539278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:08:07.541105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:08:07.541955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 591/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:08:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:08:07.547681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:07.601483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:07.658348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfcb54-580c-4701-94a9-a13b047e76dd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:07.700131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:07.700755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:07.701040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:07.701318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:07.702338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:07.702617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:07.704322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bb05a-dbd4-426b-a8e9-a2da7f86b30a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:07.715221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 575/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Aug 1 14:08:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:09.759971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce013d-75f0-4ce2-9fbe-3b0e74c68993 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:09.760759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce013d-75f0-4ce2-9fbe-3b0e74c68993 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:09.760759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce013d-75f0-4ce2-9fbe-3b0e74c68993 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:09.761885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 592/1167] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:08:09.767894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db4f9-5e53-436e-b109-9df402f2a2e0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:09.768539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db4f9-5e53-436e-b109-9df402f2a2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:09.768539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db4f9-5e53-436e-b109-9df402f2a2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:09.793007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f58db4f9-5e53-436e-b109-9df402f2a2e0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:09.794277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f58db4f9-5e53-436e-b109-9df402f2a2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:09.876817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 576/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:09.899865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce013d-75f0-4ce2-9fbe-3b0e74c68993 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:09.937654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:10.021153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce013d-75f0-4ce2-9fbe-3b0e74c68993 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:10.059286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b59e-94a6-4644-82c1-ef3bc6b60a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:10.060007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b59e-94a6-4644-82c1-ef3bc6b60a8c None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:10.060243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b59e-94a6-4644-82c1-ef3bc6b60a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:10.060517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b59e-94a6-4644-82c1-ef3bc6b60a8c None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:10.061257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b59e-94a6-4644-82c1-ef3bc6b60a8c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:10.062660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b59e-94a6-4644-82c1-ef3bc6b60a8c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:10.083239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 593/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Mon Aug 1 14:08:09 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:12.190764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5c35-e682-45c2-a1e0-39bbbac31c62 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.191308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5c35-e682-45c2-a1e0-39bbbac31c62 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.191308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5c35-e682-45c2-a1e0-39bbbac31c62 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.192747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 577/1170] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:12.198518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.199019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.199144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.218948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:12.220207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:12.293928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 594/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:12.317696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c5c35-e682-45c2-a1e0-39bbbac31c62 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.373796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:12.433700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c5c35-e682-45c2-a1e0-39bbbac31c62 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.473525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9401-795e-49a1-b8a7-8d91c2fe4599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.474088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9401-795e-49a1-b8a7-8d91c2fe4599 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.474410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9401-795e-49a1-b8a7-8d91c2fe4599 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.474658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9401-795e-49a1-b8a7-8d91c2fe4599 None admin] PATH_INFO: `/v3/domains/2edce818f5b64cfca7ad840d77e058f4` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.486683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d9401-795e-49a1-b8a7-8d91c2fe4599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2edce818f5b64cfca7ad840d77e058f4)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.488199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d9401-795e-49a1-b8a7-8d91c2fe4599 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.489925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 578/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Aug 1 14:08:12 2022] GET /identity/v3/domains/2edce818f5b64cfca7ad840d77e058f4 => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:12.496162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:12.519433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:12.605563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e463c19-d665-4965-a372-ed66c0cdd62b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:12.650928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa572ba7-ec3e-48b6-a216-aa6cac0c5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:12.651604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa572ba7-ec3e-48b6-a216-aa6cac0c5847 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:12.651877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa572ba7-ec3e-48b6-a216-aa6cac0c5847 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:12.652152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa572ba7-ec3e-48b6-a216-aa6cac0c5847 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:12.652846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa572ba7-ec3e-48b6-a216-aa6cac0c5847 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:12.654282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa572ba7-ec3e-48b6-a216-aa6cac0c5847 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:12.683768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 595/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:08:12 2022] POST /identity/v3/projects => generated 361 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:14.780761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cbd92-3d0b-4764-9bf8-7ddf2c72ccdd None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.781324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cbd92-3d0b-4764-9bf8-7ddf2c72ccdd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.781324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cbd92-3d0b-4764-9bf8-7ddf2c72ccdd None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.782685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 579/1174] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:14.787895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:14.788372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:14.788523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:14.808104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:14.809324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:14.882797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 596/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:14.908002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cbd92-3d0b-4764-9bf8-7ddf2c72ccdd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.019636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cbd92-3d0b-4764-9bf8-7ddf2c72ccdd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.022207 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:15.054034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219001b-0461-4e4f-bb54-02ad69815e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.054728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219001b-0461-4e4f-bb54-02ad69815e32 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.055552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219001b-0461-4e4f-bb54-02ad69815e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.055829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219001b-0461-4e4f-bb54-02ad69815e32 None admin] PATH_INFO: `/v3/domains/2edce818f5b64cfca7ad840d77e058f4` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.056977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219001b-0461-4e4f-bb54-02ad69815e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2edce818f5b64cfca7ad840d77e058f4)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.058424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219001b-0461-4e4f-bb54-02ad69815e32 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.060043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 580/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Aug 1 14:08:14 2022] GET /identity/v3/domains/2edce818f5b64cfca7ad840d77e058f4 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:15.065831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:15.068040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:15.172314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48d06d-716c-467c-b3a5-fba4dd358ddc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:15.208209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffed97-e8ed-4e26-9a1e-6b4b1882c47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:15.208826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffed97-e8ed-4e26-9a1e-6b4b1882c47c None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:15.209111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffed97-e8ed-4e26-9a1e-6b4b1882c47c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:15.209385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffed97-e8ed-4e26-9a1e-6b4b1882c47c None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:15.210068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffed97-e8ed-4e26-9a1e-6b4b1882c47c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:15.212056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffed97-e8ed-4e26-9a1e-6b4b1882c47c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:15.233360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 597/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:08:15 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:17.292298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcb4f2-1643-430a-8304-9119b2ec9f85 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.292872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcb4f2-1643-430a-8304-9119b2ec9f85 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.292872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcb4f2-1643-430a-8304-9119b2ec9f85 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.294189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 581/1178] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:17.299697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.300198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.300301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.318726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:17.319934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:17.375753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:17.395968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 598/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:17.423103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcb4f2-1643-430a-8304-9119b2ec9f85 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.544360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcb4f2-1643-430a-8304-9119b2ec9f85 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.581638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.582288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.582595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.582873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.593973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:08:17.599373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:08:17.600905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:08:17.602134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 582/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:08:17.606894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.707713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994484bd-78e1-4a25-9d4a-f20a0a24f138 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.743248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.743898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.744178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.744452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.745178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:17.745480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.746863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.755493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 599/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:17.761449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.793769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:17.864307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbb55e-4c49-49f2-88f3-af892bfe52d8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:17.898969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:17.899644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:17.899927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:17.900204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] PATH_INFO: `/v3/users/40387739f22d426b984cf6fca1007c5c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:17.914539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] RBAC: Authorizing `identity:get_user(user_id=40387739f22d426b984cf6fca1007c5c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:17.916052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:17.917773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 583/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/users/40387739f22d426b984cf6fca1007c5c => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:08:17.923687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:17.977486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:18.029935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9e66-f992-4b21-89ee-43684d8dd79c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.066584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.067201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.067500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.067777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.140233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.141606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.191782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:18.243356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 600/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:08:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:08:18.249478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.345984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:18.358183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12a30b-9e97-415f-bc26-e14a21dadfb7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.392438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9ca50-0a9e-420e-ad27-1f4ac7fb6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.393001 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ca50-0a9e-420e-ad27-1f4ac7fb6af0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.393281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ca50-0a9e-420e-ad27-1f4ac7fb6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.393555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ca50-0a9e-420e-ad27-1f4ac7fb6af0 None admin] PATH_INFO: `/v3/projects/72e595e343d444b4998d6cf8a68f0a5a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.394715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9ca50-0a9e-420e-ad27-1f4ac7fb6af0 None admin] RBAC: Authorizing `identity:get_project(project_id=72e595e343d444b4998d6cf8a68f0a5a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.396186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9ca50-0a9e-420e-ad27-1f4ac7fb6af0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.397875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 584/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:08:18 2022] GET /identity/v3/projects/72e595e343d444b4998d6cf8a68f0a5a => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:18.404597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:18.505147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139379e0-e1cd-47f0-b559-91b96508f371 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:18.540036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:18.540647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:18.540925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:18.541195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:18.541971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=72e595e343d444b4998d6cf8a68f0a5a, user.id=40387739f22d426b984cf6fca1007c5c) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:18.542282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:18.543724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102a90-54dc-4733-b847-1862c04bf16e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:18.549156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 601/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:08:18 2022] GET /identity/v3/role_assignments?user.id=40387739f22d426b984cf6fca1007c5c&scope.project.id=72e595e343d444b4998d6cf8a68f0a5a&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:20.552345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12aaa-1459-4299-a494-e9cdb705d4f2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.552917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12aaa-1459-4299-a494-e9cdb705d4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.552917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12aaa-1459-4299-a494-e9cdb705d4f2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.554246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 585/1186] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:20.560031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.560480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.560568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.579957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:20.581676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:20.591618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:20.660925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 602/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:20.683393 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12aaa-1459-4299-a494-e9cdb705d4f2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:20.782948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12aaa-1459-4299-a494-e9cdb705d4f2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:20.817834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:20.818383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.818662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.818932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.828896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:08:20.833760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:08:20.834841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:08:20.835918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 586/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:08:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:08:20.841123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:20.942026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226d4fa-e092-45f9-a7d9-0bc71d65b879 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:20.978438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:20.979152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:20.979492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:20.979770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:20.980491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:20.980796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:20.982159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:20.990813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 603/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:08:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:20.997711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:21.017677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:21.101174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2cd46-b2ef-46bd-96ae-be8bca947e34 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:21.135345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:21.136018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:21.136301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:21.136577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] PATH_INFO: `/v3/users/40387739f22d426b984cf6fca1007c5c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:21.137789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] RBAC: Authorizing `identity:get_user(user_id=40387739f22d426b984cf6fca1007c5c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:21.139250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:21.140920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 587/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:08:20 2022] GET /identity/v3/users/40387739f22d426b984cf6fca1007c5c => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:21.146769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:21.156736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:21.247907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7397ff-dab1-4bb2-ac27-08d1f978388c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:21.281937 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeba28f-6029-4961-9301-09b49f5799d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:21.282596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeba28f-6029-4961-9301-09b49f5799d9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:21.282875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeba28f-6029-4961-9301-09b49f5799d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:21.283151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeba28f-6029-4961-9301-09b49f5799d9 None admin] PATH_INFO: `/v3/projects/72e595e343d444b4998d6cf8a68f0a5a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:21.284344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeba28f-6029-4961-9301-09b49f5799d9 None admin] RBAC: Authorizing `identity:get_project(project_id=72e595e343d444b4998d6cf8a68f0a5a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:21.285783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeba28f-6029-4961-9301-09b49f5799d9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:21.287481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 604/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:08:21 2022] GET /identity/v3/projects/72e595e343d444b4998d6cf8a68f0a5a => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:21.294280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:21.389311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:21.404641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085e486-1868-47ee-9af8-2cbafaeafe28 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:21.438071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a62ca0-dddc-42ae-a65a-c79e3c9234d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:21.438735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a62ca0-dddc-42ae-a65a-c79e3c9234d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:21.439017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a62ca0-dddc-42ae-a65a-c79e3c9234d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:21.439291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a62ca0-dddc-42ae-a65a-c79e3c9234d8 None admin] PATH_INFO: `/v3/projects/72e595e343d444b4998d6cf8a68f0a5a/users/40387739f22d426b984cf6fca1007c5c/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:21.447194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a62ca0-dddc-42ae-a65a-c79e3c9234d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=72e595e343d444b4998d6cf8a68f0a5a, user_id=40387739f22d426b984cf6fca1007c5c, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:21.448816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a62ca0-dddc-42ae-a65a-c79e3c9234d8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:21.463514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 588/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:08:21 2022] PUT /identity/v3/projects/72e595e343d444b4998d6cf8a68f0a5a/users/40387739f22d426b984cf6fca1007c5c/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:23.488461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef1649-80d6-4095-bff3-383092dcfb7c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.489035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef1649-80d6-4095-bff3-383092dcfb7c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.489035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef1649-80d6-4095-bff3-383092dcfb7c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.490345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 605/1193] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:23.495746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.496233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.496367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.515690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:23.517381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:23.587219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 589/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:08:23.611328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef1649-80d6-4095-bff3-383092dcfb7c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:23.691272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:23.719141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef1649-80d6-4095-bff3-383092dcfb7c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:23.753934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:23.754511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.754934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.755066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.766514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:08:23.771444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:08:23.772567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:08:23.774101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 606/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:08:23.779437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:23.869673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:23.893807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fa68e-7d19-498c-afce-d3e078106d1a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:23.928455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:23.929074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:23.929390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:23.929665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:23.930390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:23.930691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:23.932105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:23.940168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 590/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:23.946547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.046184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed564f88-9ae2-46c9-8bcd-4a6f1ec66817 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.079766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.080401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.080681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.080954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] PATH_INFO: `/v3/users/40387739f22d426b984cf6fca1007c5c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.082121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] RBAC: Authorizing `identity:get_user(user_id=40387739f22d426b984cf6fca1007c5c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.083582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.085199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 607/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Aug 1 14:08:23 2022] GET /identity/v3/users/40387739f22d426b984cf6fca1007c5c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:24.091681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.192508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24923a7-a57e-4b9f-b60c-30e4249ddfc0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.226464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.227082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.227367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.227686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.236304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:24.296039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.297586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.396353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 591/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:08:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:08:24.402310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.405194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:24.507011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ef69e-151a-4aa6-9390-7ad7c075bb78 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.542266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acaecc-d123-4eb0-bf5a-0a2e40abb152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.543024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acaecc-d123-4eb0-bf5a-0a2e40abb152 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.543180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acaecc-d123-4eb0-bf5a-0a2e40abb152 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.543479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acaecc-d123-4eb0-bf5a-0a2e40abb152 None admin] PATH_INFO: `/v3/projects/72e595e343d444b4998d6cf8a68f0a5a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.544657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acaecc-d123-4eb0-bf5a-0a2e40abb152 None admin] RBAC: Authorizing `identity:get_project(project_id=72e595e343d444b4998d6cf8a68f0a5a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.546087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acaecc-d123-4eb0-bf5a-0a2e40abb152 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.547858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 608/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Aug 1 14:08:24 2022] GET /identity/v3/projects/72e595e343d444b4998d6cf8a68f0a5a => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:24.554030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:24.593418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:24.653919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72eeef-f9b4-4e9b-8fb6-254bc482b345 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:24.687234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:24.687897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:24.688179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:24.688453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:24.689250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=72e595e343d444b4998d6cf8a68f0a5a, user.id=40387739f22d426b984cf6fca1007c5c) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:24.689641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:24.691051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f67cbb-f60e-4c57-866f-b714d304c174 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:24.697914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 592/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:08:24 2022] GET /identity/v3/role_assignments?user.id=40387739f22d426b984cf6fca1007c5c&scope.project.id=72e595e343d444b4998d6cf8a68f0a5a&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:34.537875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=65 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:34.695450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=49 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:22.226531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be3015-61ff-4e4c-b8ac-87d9ceb48c01 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.226531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be3015-61ff-4e4c-b8ac-87d9ceb48c01 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.226531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be3015-61ff-4e4c-b8ac-87d9ceb48c01 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.227616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 609/1201] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:09:22.234009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.234494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.234805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.258328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:22.260058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.388355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 593/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:09:22.423168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be3015-61ff-4e4c-b8ac-87d9ceb48c01 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.613324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be3015-61ff-4e4c-b8ac-87d9ceb48c01 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.667084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.667833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.668140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.668416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.683745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.685525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:22.708097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:22.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:22.712412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:22.712412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:22.712412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:22.712412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:22.712412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 610/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:22.716438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.858635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400ccb-4223-4f35-93d1-d23af1916068 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.904082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:22.910841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.912564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.913052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.913539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.914903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:22.915473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.918316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce72fe8-eed5-4e75-923b-4ce753ee8ae7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.938726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 594/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:22.940218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.037021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:23.105632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ab4f0-c916-4425-b4df-6b2165573d00 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.162424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ab89c-c60c-4eee-9d2f-33972bbce894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.163116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab89c-c60c-4eee-9d2f-33972bbce894 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.163514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab89c-c60c-4eee-9d2f-33972bbce894 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.163929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab89c-c60c-4eee-9d2f-33972bbce894 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.164850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ab89c-c60c-4eee-9d2f-33972bbce894 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.166744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ab89c-c60c-4eee-9d2f-33972bbce894 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.195299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 611/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/users => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:25.646940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4f4c5-dbdc-40a8-b5a6-ccfb8c89f3b1 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.647627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4f4c5-dbdc-40a8-b5a6-ccfb8c89f3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.647627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4f4c5-dbdc-40a8-b5a6-ccfb8c89f3b1 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.649438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 595/1206] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:25.660577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.660577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.660577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.679981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.681267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.778866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 612/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:25.805896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4f4c5-dbdc-40a8-b5a6-ccfb8c89f3b1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.944801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4f4c5-dbdc-40a8-b5a6-ccfb8c89f3b1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.987853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.988905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.989460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.989747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.001806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:09:26.007628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:26.008806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:26.009666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 596/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:26.015276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.068430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.144341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a83ce0-872e-4bd4-9a30-71c43449c461 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.189350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.190172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.190458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.190757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.191723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:26.192102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.194039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.207569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 613/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:26.211436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.279248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.344856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3081c-d1c5-46f7-ad66-039af842bb06 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.401201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.402164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.402604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.403048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.421327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.423195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:26.444812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:26.447310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:26.448747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:26.448747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:26.448747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:26.448747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:26.448747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 597/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:26.453308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.581651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.584941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa385e1-9528-43b7-a095-3e912b9086cc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.624372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.624974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.625258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.625532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.626220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:26.626523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.627930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.646142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 614/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:26.649218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.775644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988589c-870f-4b8d-9005-d076d142da5d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.777534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.822299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.822299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.822299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.822299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.830406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.832939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:09:26.845923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:26.847355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:26.853837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 598/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:26.854478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.972667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01275eee-c282-4181-b3a1-3f92624760ee None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.016791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.017662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.018062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.018461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.019482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:27.019905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.021608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.042584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 615/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:27.049181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.192221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3026a-b062-4b4c-a52a-bc48f377f0ed None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.233245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.233884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.234306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.234567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.278381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.347356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.350094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.501655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 599/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:09:27.503001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.529282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.630969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c69065-5f21-4905-8676-011fec40c329 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.673912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.674600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.674910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.675329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.690941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.693423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:27.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:27.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:27.719314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:27.719314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:27.719314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:27.719314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:27.719314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 616/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:27.720428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.806414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.861809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af55aa-ff09-4ea4-b4b7-03dc051f3ef6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.910056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.910685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.910967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.911237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.911968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:27.912274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.914162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.926635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 600/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:27.932984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.025860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.069739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130256d3-357d-449a-ba4c-c5364e0fad1c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.114144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.115598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.115886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.116260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.125448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.127362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:09:28.142161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:28.143682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:28.144993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 617/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:28.149964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.281958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04d67d-3928-44d8-95d4-bcc54090aba4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.310434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.326574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.327128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.327412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.327713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.328514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:28.328907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.330430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8310e4-b9ab-4211-a294-861873044a4a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.351501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 601/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:28.355023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.475446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae0013-6c54-48bd-8f92-2e4d5084742e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.512266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.527141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.528203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.528658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.529114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.530578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=324fc1c2ea18420eb01d8b65683818ff) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:28.531072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.533602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60719a-eda1-488c-8d0c-3453d8e0a604 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.549350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 618/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/role_assignments?user.id=324fc1c2ea18420eb01d8b65683818ff&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:30.824198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34c8a-b23c-4e15-9704-cd0ff27a29d0 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.824788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34c8a-b23c-4e15-9704-cd0ff27a29d0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.824926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34c8a-b23c-4e15-9704-cd0ff27a29d0 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.826437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 602/1220] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:30.831662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.832110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.832246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.851157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:30.852401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.930733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 619/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:30.953451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34c8a-b23c-4e15-9704-cd0ff27a29d0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.087054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34c8a-b23c-4e15-9704-cd0ff27a29d0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.126912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.127842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.129389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.129389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.144831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:09:31.149942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:31.151364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:31.152407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 603/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:09:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:31.157772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.293864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357a47-c7b7-4e82-84a7-176ad0551c2d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.326061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:31.345291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.346217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.346640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.347321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.348516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:31.349578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.352190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.368129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 620/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:31.374365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.433760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:31.500492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdb6d4-c92a-4f1b-9854-10860c4e48bb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.541096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.542039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.542327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.542614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.555011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.556519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:31.572322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:31.573384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:31.574449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:31.574449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:31.574449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:31.574449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:31.574449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 604/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:31.580132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.726817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6fbce-16ab-4a08-91a3-dd711eab0d3f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.764856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.765477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.765758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.766030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.766713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:31.767019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.768454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.781921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 621/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:31.788582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:31.900847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:31.909217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a0a47-7644-40da-b382-d5c9c17a728b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.946686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.947335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.947659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.947938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.956765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.959481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:09:31.971087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:31.972526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:31.974480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 605/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:31.979369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.037183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:32.094214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dff60-e2ac-4204-beb7-4f4a280e4346 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.131006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.131736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.132079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.132449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.133364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:32.133775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.135285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.157136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 622/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Aug 1 14:09:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:32.163782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.305644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f78297-3b47-4e91-827c-1537e26f1a96 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.344246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.345129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.345598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.345991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.360462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.362140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:32.380524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:32.381654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:32.382658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:32.382658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:32.382658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:32.382658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:32.382658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 606/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:32.387639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.494687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfad0d-1d99-41d2-aa54-7d0f54165f92 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.497416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:32.538386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.539165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.539591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.539866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.540561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:32.540862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.542261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.559715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 623/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:32.561153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.571554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:32.694849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998377-43d2-40ca-a97f-5c2acc87d003 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.735579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.736219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.736499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.736772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.744913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.746542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:09:32.759786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:32.760825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:32.761706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 607/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:32.767263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.881179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60686cd6-f24c-4c23-9f3e-c0fbfec0c5fd None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.920199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.920854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.921251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.921656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.922686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:32.923091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.925012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a40c3-f8d0-46c9-94b7-8d0826b09089 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.948346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 624/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:32.949503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.038241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:33.081200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a0ed3-7078-41a5-9ab0-a9f8ec2469df None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.129478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc2d38-12d7-441d-bf50-5b11ea5888cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.130230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc2d38-12d7-441d-bf50-5b11ea5888cb None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.130522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc2d38-12d7-441d-bf50-5b11ea5888cb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.130800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc2d38-12d7-441d-bf50-5b11ea5888cb None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/324fc1c2ea18420eb01d8b65683818ff/roles/d011b6eea3fb4e1da4b8d9147d942725` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.148957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc2d38-12d7-441d-bf50-5b11ea5888cb None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=324fc1c2ea18420eb01d8b65683818ff, role_id=d011b6eea3fb4e1da4b8d9147d942725)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.150473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc2d38-12d7-441d-bf50-5b11ea5888cb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.171645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 608/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:09:32 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/324fc1c2ea18420eb01d8b65683818ff/roles/d011b6eea3fb4e1da4b8d9147d942725 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:35.218797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a6f52-5829-44d8-a99f-bb27e23088de None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:35.219434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a6f52-5829-44d8-a99f-bb27e23088de None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:35.219434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a6f52-5829-44d8-a99f-bb27e23088de None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:35.220854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 625/1233] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:35.227537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:35.228052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:35.228159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:35.249997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:35.251310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:35.345676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 609/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:35.371703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a6f52-5829-44d8-a99f-bb27e23088de None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:35.403046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:35.502047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a6f52-5829-44d8-a99f-bb27e23088de None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:35.545458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:35.546250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:35.546692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:35.547115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:35.558557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:09:35.563587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:35.564790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:35.565679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 626/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Aug 1 14:09:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:35.571332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:35.636846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:35.711799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45795dc7-503b-4393-9574-0ed475ba987e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:35.751911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:35.752596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:35.752822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:35.753096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:35.753941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:35.754249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:35.756187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:35.766520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 610/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Aug 1 14:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:35.773113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:35.902711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:35.923737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b43f2-d829-4d5e-97c8-bd47b6ac3f3a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:35.961605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:35.962254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:35.962537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:35.962829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:35.975013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:35.976656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:35.995054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:35.996282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:35.997336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:35.997336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:35.997336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:35.997336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:35.997336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 627/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:36.003245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:36.152519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d153f8-734e-4a46-a060-2129b46a51e6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:36.163417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:36.202724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:36.203587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:36.203878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:36.204158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:36.204879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:36.205185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:36.206588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:36.220220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 611/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:36.227101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:36.356023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b343a2-a9fa-4e24-b6bf-bdf189c9b865 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:36.393220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:36.393714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:36.394076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:36.394277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:36.397280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:36.402859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:36.404617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:09:36.417160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:36.418341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:36.419662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 628/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:36.424474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:36.535022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580218c-6175-411c-8bd1-8e05b9ee18f7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:36.576886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:36.578141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:36.578381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:36.578572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:36.579591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:36.580025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:36.581769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:36.604597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 612/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Aug 1 14:09:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:36.612281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:36.736963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69213a-b0b7-41a3-999f-c443fe156fff None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:36.776666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:36.777325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:36.777626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:36.777917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:36.856541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:36.866796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:36.868243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:36.991129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 629/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:09:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:09:36.997613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:37.028023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:37.119331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec712-92a6-4296-bd4f-a0f46e484869 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.181313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.181995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.182438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.182701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.199773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.201974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:37.224564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:37.225692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:37.226792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:37.226792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:37.226792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:37.226792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:37.226792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 613/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:37.229978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:37.353522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:37.360765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a6897-fe78-43f1-9c7c-d02575589729 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.407138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.408084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.408557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.409062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.410605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:37.410772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.414107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:37.431491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 630/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:37.437796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:37.567977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:37.592879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d777fe8-d0a6-4c90-a7c3-eed478c9d5b1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.633264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.634005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.634311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.634564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.642561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.644147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:09:37.658110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:37.659262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:37.660196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 614/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:37.665299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:37.798513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716b751-110f-4d7f-bb4b-6c7cb5f2bf85 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:37.841192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:37.841819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:37.842097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:37.842499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:37.843333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:37.843684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:37.845152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ceb72b-71a1-4d5c-960e-e69c2408009f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:37.859237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 631/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:37.866886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:38.012346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12ff1-f1f4-48db-98c2-d7961ed0a7ed None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:38.051050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:38.052396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:38.052786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:38.053084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:38.054046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] RBAC: Adding query filter params (role.id=d011b6eea3fb4e1da4b8d9147d942725, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=324fc1c2ea18420eb01d8b65683818ff) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:38.054432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:38.056216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d67931-fe30-4799-9e4a-f42b36f8f9f5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:38.064836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 615/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:09:37 2022] GET /identity/v3/role_assignments?user.id=324fc1c2ea18420eb01d8b65683818ff&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=d011b6eea3fb4e1da4b8d9147d942725 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:40.374616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644918-c889-4d47-9745-a72dd411b6c4 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.375186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644918-c889-4d47-9745-a72dd411b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.375271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644918-c889-4d47-9745-a72dd411b6c4 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.376685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 632/1247] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:40.383182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.383780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.383780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.390692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:40.405506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:40.406754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.499655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 616/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:09:40.532845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea644918-c889-4d47-9745-a72dd411b6c4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.556835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:40.677997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea644918-c889-4d47-9745-a72dd411b6c4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.733134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.733783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.734069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.734340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.747013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:09:40.752261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:40.754121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:40.755797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 633/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:40.763852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.906538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ec531-8d85-4383-8887-f679bcc13d3c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.955091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:40.961250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.962180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.962491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.962810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.963553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:40.963868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.965289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.977739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 617/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:40.984256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.073823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:41.101389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b55e-6996-4d40-be65-eb8812e89f69 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.139343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.140020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.140313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.140586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.153359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.154803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:41.171900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:41.173023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:41.174093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:41.174093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:41.174093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:41.174093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:41.174093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 634/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:41.181013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.296732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47457e-952e-4183-8b7c-402e9809214f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.341882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.342529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.342814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.343093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.343852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:41.344164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.345572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.361646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 618/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:41.369212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.487772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb1716-6c71-40ff-90ae-daf3f4c41dae None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.521078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:41.530332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.531138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.531557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.531948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.543584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.545088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:09:41.561639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:41.562936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:41.564031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 635/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:41.571122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.643194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:41.719898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a11a-737a-41a3-be14-7bbb618ea9f9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.771403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.772074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.772349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.772623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.773440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:41.773746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.775190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.798957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 619/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:41.807498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.931915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce93e65-e751-4be1-be24-8ee3e6450d9b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.981308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.982815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.983632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.984185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.068238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:42.086645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.088580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.257088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 636/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:09:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:09:42.258703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.390611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eee48d-ae49-4772-9aef-395326e65639 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.408509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:42.434229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.435072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.435487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.435812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.450503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.452342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:42.470585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:42.472578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:42.474270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:42.474270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:42.474270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:42.474270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:42.474270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 620/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:42.480102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.607926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:42.634799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35148791-ea1a-469c-9112-1337cc54087c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.686090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.686719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.687003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.687280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.688028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:42.688339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.689862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.705437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 637/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:42.711284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.851011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b292045-f3f3-4f22-98e2-70c9fe2e36a0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.896864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.897489 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.897694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.897994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.906067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.907991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:09:42.924426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:42.925454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:42.926352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 621/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:42.932535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.051959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f40ce-e04a-4610-8a7a-9959f5002a46 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.092471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.093145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.093498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.093763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.095160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:43.095160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.096527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd417acf-8a8d-4b22-900b-35ef9bdc4b6a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.115781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 638/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:43.124703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.161297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:43.271961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca9764-9329-4650-b79b-95a6f1444286 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.319576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.320361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.320630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.320937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.321934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=324fc1c2ea18420eb01d8b65683818ff) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:43.322244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.323806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad9088-b4a6-42b8-90b5-7e9f51dcca91 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.330993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 622/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:09:43 2022] GET /identity/v3/role_assignments?user.id=324fc1c2ea18420eb01d8b65683818ff&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:45.484465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15dfac-a1b7-45f1-adf0-cceef3c4b434 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.485002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15dfac-a1b7-45f1-adf0-cceef3c4b434 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.485002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15dfac-a1b7-45f1-adf0-cceef3c4b434 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.486389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 639/1261] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:09:45.492364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.492868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.492976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.515218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:45.516498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.599128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 623/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:09:45.621282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15dfac-a1b7-45f1-adf0-cceef3c4b434 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:45.632649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:45.742717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15dfac-a1b7-45f1-adf0-cceef3c4b434 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:45.778971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:45.779648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.779935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.780213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.790765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:09:45.795773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:45.796957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:45.802871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 640/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:09:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:45.803780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:45.822019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:45.910460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336162-0ec0-4e6c-a26b-1a607d7d2d85 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:45.946419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:45.947050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.947337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.947655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.948390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:45.948699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:45.950122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:45.959347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 624/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:09:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:45.965946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.050326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.077303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b56f5-6684-4888-989e-edfb54712c45 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.111143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.111822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.112101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.112372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.124448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.125901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:46.141654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:46.142832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:46.144243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:46.144243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:46.144243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:46.144243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:46.144243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 641/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:46.149934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.251793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.271986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c8d0e-aaaf-40c7-af1b-7c2f58506c95 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.307883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.308420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.308705 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.308979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.309686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:46.309989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.311376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.323929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 625/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:46.330122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.434957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8bb9-f30c-4afb-b590-f7541c529f34 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.455919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.470535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.471186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.471494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.472055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.480555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.482021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:09:46.494454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:46.496059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:46.497114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 642/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:46.501673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.605594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9197b-dcdf-440e-baed-cf0820475fda None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.640752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.641388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.641679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.641956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.642775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:46.643083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.644579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.654737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.665116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 626/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:46.670931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.774501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06762e34-1af2-44f3-8653-eabd32d9b31f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.808938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.809574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.809853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.810126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.821123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.822588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:46.837683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:46.838888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:46.839908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:46.839908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:46.839908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:46.839908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:46.839908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 643/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:46.844915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.959962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ccd7f-af7a-46fd-9d92-d96677760685 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.999112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.999769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.000050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.000324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.001029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:47.001331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.002705 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.015407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 627/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:47.021974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.055985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.128149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4576690-ce3d-47d2-b697-dfd1663d39d8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.172927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.173861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.174227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.174601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.187374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.187374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:09:47.199494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:47.200741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:47.201905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 644/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:47.206684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.272217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.318593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7a0d8-2536-410f-956f-8e57783bce0f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.353713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.354334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.354621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.354902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.355766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:47.358118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.358118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437a3ba-52cf-42ac-91db-d803c756283c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.371662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 628/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:47.377745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.488051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.499970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12675-b697-4e75-b704-9e5d9945bc2c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.542001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b0cef-79bd-43db-8ef2-b939bb3f2a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.542595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b0cef-79bd-43db-8ef2-b939bb3f2a33 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.542881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b0cef-79bd-43db-8ef2-b939bb3f2a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.543216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b0cef-79bd-43db-8ef2-b939bb3f2a33 None admin] PATH_INFO: `/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/324fc1c2ea18420eb01d8b65683818ff/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.545145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b0cef-79bd-43db-8ef2-b939bb3f2a33 None admin] RBAC: Authorizing `identity:create_grant(project_id=f106ff2ec5cb4ecda3cd56c2391bc200, user_id=324fc1c2ea18420eb01d8b65683818ff, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.546749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b0cef-79bd-43db-8ef2-b939bb3f2a33 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.561066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 645/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Aug 1 14:09:47 2022] PUT /identity/v3/projects/f106ff2ec5cb4ecda3cd56c2391bc200/users/324fc1c2ea18420eb01d8b65683818ff/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:49.605457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825b53-af35-4710-aac5-dc6e0f695503 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.606056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825b53-af35-4710-aac5-dc6e0f695503 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.606056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825b53-af35-4710-aac5-dc6e0f695503 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.607368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 629/1274] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:49.613177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.614204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.614303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.633948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:49.635179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:49.704438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 646/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:49.728162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61825b53-af35-4710-aac5-dc6e0f695503 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.834086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61825b53-af35-4710-aac5-dc6e0f695503 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:49.851742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:49.869977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:49.870528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.870811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.871087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.882516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:09:49.887589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:49.888691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:49.894724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 630/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Aug 1 14:09:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:49.895667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.978092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:49.997496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2923a2c-5234-4199-823a-c09a4c7ba1c6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.031957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.032581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.032862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.033139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.033864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:50.034167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.036258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.044933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 647/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Aug 1 14:09:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:50.050786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.157361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358fd3a-0446-483f-8e98-cce63497c5d7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.204727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.206127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.206422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.206698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.218026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.219690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:50.236517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:50.237547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:50.239580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:50.239580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:50.239580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:50.239580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:50.239580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 631/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:50.244238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.346618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2a8ba-4079-45ac-82bf-7018969eca7f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.381934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.382558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.382841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.383116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.383847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:50.384150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.385518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.398082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 648/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:50.404007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.439100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.519826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cb32e-2e54-452e-91e1-a495ad5aac62 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.556613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.557265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.557546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.557813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.566478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.568638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:09:50.586206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:50.587230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:50.588223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 632/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:50.594128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.598419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.702860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c322-d1d1-40c5-8fe7-e94b92859a5d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.738197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.739386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.739700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.739975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.740774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:50.741084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.742513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.761155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 649/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:50.767448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.870403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.880895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1fd36-c1d7-4558-bb1a-d83ea1522ae3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.917676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.918309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.918590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.918879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.993724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.995128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.094979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.107534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 633/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:09:51.113493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.184630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.227333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95f966-baf6-4750-aaf2-30f6841d01ce None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.262490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.263141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.263424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.263744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.274881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.276394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:09:51.292604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:51.293990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:51.295328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:09:51.295328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:09:51.295328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:09:51.295328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:51.295328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 650/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:51.299744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.408802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe969e9-e0f2-4ab7-bc6b-a58211c6f08b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.446068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.446697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.446981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.447260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.447995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:51.448304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.449687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.462283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 634/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:51.467989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.577450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22191ff2-18ad-420a-856d-c5c8c37abcca None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.592829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.623648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.624311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.624596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.624873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.632903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.634381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:09:51.647825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:51.649742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 01 14:09:51.651192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 651/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:51.655634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.722403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.780286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad42c-9c03-4d67-9656-e70af19432d3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.823889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.824517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.827173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.827173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.827173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:51.827173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.827602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a6d7-e41f-485a-b028-7b88b60972ab None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.841247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 635/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:51.847742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.951164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6080cf-bc04-419c-9d47-4e97f336ebc3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.985736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.986343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.986620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.986898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.987699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] RBAC: Adding query filter params (role.id=92897736a4464cec9c4a1b3a24b138b1, scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200, user.id=324fc1c2ea18420eb01d8b65683818ff) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:51.988007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.989479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdc0a1-0857-4bb1-8b27-7154b04d6612 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.995605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 652/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/role_assignments?user.id=324fc1c2ea18420eb01d8b65683818ff&scope.project.id=f106ff2ec5cb4ecda3cd56c2391bc200&role.id=92897736a4464cec9c4a1b3a24b138b1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:54.100315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a339-892c-4021-913e-4e0e61d69a22 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.100894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a339-892c-4021-913e-4e0e61d69a22 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.100894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a339-892c-4021-913e-4e0e61d69a22 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.102243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 636/1288] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:54.108354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.108942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.108942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.127555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:54.128788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.164240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.208106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 653/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:54.230495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a339-892c-4021-913e-4e0e61d69a22 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.341401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a339-892c-4021-913e-4e0e61d69a22 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.354308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.384389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.385041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.385329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.385608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.386251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.387670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:54.397651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:54.398978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:54.399899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 637/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:09:54.405069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.512195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f17562-c2b4-4419-b89c-e9b6b5ef1587 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.549258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.549879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.550163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.550440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.551156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:54.551490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.552867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366059a4-17d3-4609-a64c-982d04fbc736 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.558123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 654/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:54.564018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.679910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d6163-f4d8-4a65-8bd7-a3febc1daf2e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.719245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.719898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.720183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.720458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.721155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:54.721466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.723226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9f1d7-25da-4047-a0cf-b466a7a4847e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.728524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 638/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:56.790088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274d5cb-73d0-479e-9333-d8c7454c0c86 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.790661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274d5cb-73d0-479e-9333-d8c7454c0c86 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.790661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274d5cb-73d0-479e-9333-d8c7454c0c86 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.792138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 655/1293] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:56.797778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728e42e-7be3-42fc-9945-201304e493eb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.798319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728e42e-7be3-42fc-9945-201304e493eb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.798319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728e42e-7be3-42fc-9945-201304e493eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.818322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1728e42e-7be3-42fc-9945-201304e493eb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.819599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1728e42e-7be3-42fc-9945-201304e493eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.854842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:56.896215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 639/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:56.919392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274d5cb-73d0-479e-9333-d8c7454c0c86 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.956190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:57.036178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274d5cb-73d0-479e-9333-d8c7454c0c86 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.072131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223c687-f64b-4f63-832f-51e3c7739d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.072749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223c687-f64b-4f63-832f-51e3c7739d9a None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.073030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223c687-f64b-4f63-832f-51e3c7739d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.073303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223c687-f64b-4f63-832f-51e3c7739d9a None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.074004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223c687-f64b-4f63-832f-51e3c7739d9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.075378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223c687-f64b-4f63-832f-51e3c7739d9a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.092721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 656/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/services => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:59.266772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51f869-52cb-4958-8a8d-cd3230252aef None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.267346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51f869-52cb-4958-8a8d-cd3230252aef None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.267346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51f869-52cb-4958-8a8d-cd3230252aef None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.268737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 640/1296] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:59.275519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.276072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.276387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.297099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:59.298417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:59.388536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 657/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:09:59 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:09:59.412761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51f869-52cb-4958-8a8d-cd3230252aef None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:59.534511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51f869-52cb-4958-8a8d-cd3230252aef None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:59.574849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:59.575754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.575925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.576076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.576718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:59.578125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:09:59.587799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:09:59.589086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:09:59.590141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:09:59.591816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 641/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:09:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:09:59.595272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:59.622302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:59.713984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159ee8b-55d7-4521-a5a3-f1a6f17812a3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:59.756275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:59.756903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.757183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.757464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.758153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:59.758524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:59.759942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:59.766087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 658/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Aug 1 14:09:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:59.772432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:59.810559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:59.889824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644264f-5d3f-4504-a0f6-92d0ebf3aad3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:59.926542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:59.927154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:59.927435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:59.927731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:59.928497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] RBAC: Adding query filter params (interface=public, service_id=a9e361b2c677486c8ffa3d59f6d3552e, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:59.928800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:59.930186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3259ae-fd17-4d05-85ac-0eb19861ce52 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:59.935929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 642/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Aug 1 14:09:59 2022] GET /identity/v3/endpoints?service_id=a9e361b2c677486c8ffa3d59f6d3552e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:59.941804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:00.045114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:00.066136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d51bdd-b6a8-4e46-a78e-f73a6a19f26e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:00.103334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7d2d9-3d23-4b3b-b977-2368237d6c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:00.103987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7d2d9-3d23-4b3b-b977-2368237d6c47 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:00.104274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7d2d9-3d23-4b3b-b977-2368237d6c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:00.104551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7d2d9-3d23-4b3b-b977-2368237d6c47 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:00.105211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7d2d9-3d23-4b3b-b977-2368237d6c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:00.106601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7d2d9-3d23-4b3b-b977-2368237d6c47 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:00.114363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 659/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Aug 1 14:09:59 2022] GET /identity/v3/services => generated 2191 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:02.174168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccec1df-01ec-4505-ac7a-6221e5bb874c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.174743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccec1df-01ec-4505-ac7a-6221e5bb874c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.174743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccec1df-01ec-4505-ac7a-6221e5bb874c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.176907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 643/1302] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:10:02.182331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.182819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.182910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.203429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:02.204707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:02.275089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 660/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:10:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:10:02.298748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccec1df-01ec-4505-ac7a-6221e5bb874c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:02.403973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:02.412342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccec1df-01ec-4505-ac7a-6221e5bb874c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:02.447118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:02.447706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.447995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.448272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.448907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:02.450282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:10:02.462159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:10:02.463407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 01 14:10:02.464527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 644/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Aug 1 14:10:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:10:02.469559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:02.559440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:02.584567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949985-87bb-432e-9237-4fbfa3a8b78f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:02.620761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:02.621395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.621676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.621955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.622645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:02.622960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:02.624387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32ad93-87c9-4a53-9bba-923fb5a28a84 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:02.629959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 661/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Aug 1 14:10:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:10:02.635951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:02.751247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332fc54-c6b3-4701-9967-b013801431d3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:02.786563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77f065-4396-43a1-904c-67ea0aa95aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:02.787183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f065-4396-43a1-904c-67ea0aa95aed None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:02.787492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f065-4396-43a1-904c-67ea0aa95aed None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:02.787763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f065-4396-43a1-904c-67ea0aa95aed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:02.788495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77f065-4396-43a1-904c-67ea0aa95aed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:02.789869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77f065-4396-43a1-904c-67ea0aa95aed None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:02.811871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 645/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:12.626751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=30 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:12.807068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:10.220960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50a431-b010-41b1-ade2-83fee27f6330 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:10.222357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50a431-b010-41b1-ade2-83fee27f6330 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:10.222856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50a431-b010-41b1-ade2-83fee27f6330 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:10.224566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 662/1307] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:11:10.230826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c93b2-48c1-481b-be4a-18ab38a0a635 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:10.231833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c93b2-48c1-481b-be4a-18ab38a0a635 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:10.232122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c93b2-48c1-481b-be4a-18ab38a0a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:10.347360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-658c93b2-48c1-481b-be4a-18ab38a0a635 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:10.349069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-658c93b2-48c1-481b-be4a-18ab38a0a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:10.485432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 646/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:10.551247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e135a-ecd9-4be2-a3cc-7b6ea7e8b5e2 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:10.553760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e135a-ecd9-4be2-a3cc-7b6ea7e8b5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:10.555964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e135a-ecd9-4be2-a3cc-7b6ea7e8b5e2 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:10.560091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 663/1309] 10.222.0.17 () {54 vars in 965 bytes} [Mon Aug 1 14:11:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:10.566704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906001-651c-491d-a790-92ec639e5da8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:10.567838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906001-651c-491d-a790-92ec639e5da8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:10.568371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906001-651c-491d-a790-92ec639e5da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:10.697060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-36906001-651c-491d-a790-92ec639e5da8 None None] MFA Rules not processed for user `771f49a069014fdc956e073c2d0a0fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:10.698868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-36906001-651c-491d-a790-92ec639e5da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:10.734066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:10.811861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 647/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Mon Aug 1 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:10.821533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e135a-ecd9-4be2-a3cc-7b6ea7e8b5e2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:10.824294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ee8e135a-ecd9-4be2-a3cc-7b6ea7e8b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:10.993853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e135a-ecd9-4be2-a3cc-7b6ea7e8b5e2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:11.092895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:11.093533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:11.093816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:11.094113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:11.096736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:11.168313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:11.254394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:11.256621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9b47f-3832-4e54-8e82-2ed6ac097092 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:11.284071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 664/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:11:13.773355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adabcb-4d32-4a08-a836-24548f7500a4 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:13.773926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adabcb-4d32-4a08-a836-24548f7500a4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:13.773926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adabcb-4d32-4a08-a836-24548f7500a4 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:13.775337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 648/1312] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:13.781327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb373a-48ab-427a-9091-b433a44c8f48 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:13.781890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb373a-48ab-427a-9091-b433a44c8f48 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:13.781890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb373a-48ab-427a-9091-b433a44c8f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:13.885006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6fbb373a-48ab-427a-9091-b433a44c8f48 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:13.886612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6fbb373a-48ab-427a-9091-b433a44c8f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:14.010843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 665/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:14.050475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adabcb-4d32-4a08-a836-24548f7500a4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:14.052390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-75adabcb-4d32-4a08-a836-24548f7500a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:14.174599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adabcb-4d32-4a08-a836-24548f7500a4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:14.263887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:14.268202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:14.269279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:14.270308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:14.270666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:14.272556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:14.406807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:14.408521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37845b45-7377-4d74-bcaf-66a671119a80 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:14.429121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 649/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:16.870050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b8a9-820c-4801-a78a-fb2bce9a04ee None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:16.870652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b8a9-820c-4801-a78a-fb2bce9a04ee None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:16.870652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b8a9-820c-4801-a78a-fb2bce9a04ee None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:16.872091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 666/1315] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:16.878647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ddc65-4353-4f3a-a3e8-be62b79b8721 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:16.879196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ddc65-4353-4f3a-a3e8-be62b79b8721 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:16.879196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ddc65-4353-4f3a-a3e8-be62b79b8721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:16.960600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-456ddc65-4353-4f3a-a3e8-be62b79b8721 None None] MFA Rules not processed for user `b0b3f354b8b6415a9dda8dd637e5335e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:16.961949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-456ddc65-4353-4f3a-a3e8-be62b79b8721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:17.019874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:17.067551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 650/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:17.131449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a905b0-6ff3-45c5-8b61-f1a2b0209b18 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:17.131769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a905b0-6ff3-45c5-8b61-f1a2b0209b18 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:17.132051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a905b0-6ff3-45c5-8b61-f1a2b0209b18 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:17.133430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 667/1317] 10.222.0.17 () {54 vars in 965 bytes} [Mon Aug 1 14:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:17.139855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004741d-f0b0-4276-9ccd-729d158c6d1a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:17.140139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004741d-f0b0-4276-9ccd-729d158c6d1a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:17.140412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004741d-f0b0-4276-9ccd-729d158c6d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:17.226745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b004741d-f0b0-4276-9ccd-729d158c6d1a None None] MFA Rules not processed for user `771f49a069014fdc956e073c2d0a0fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:17.227940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b004741d-f0b0-4276-9ccd-729d158c6d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:17.310317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 651/1318] 10.222.0.17 () {58 vars in 1062 bytes} [Mon Aug 1 14:11:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:17.319566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a905b0-6ff3-45c5-8b61-f1a2b0209b18 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:17.321129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-12a905b0-6ff3-45c5-8b61-f1a2b0209b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:17.331651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:17.436168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a905b0-6ff3-45c5-8b61-f1a2b0209b18 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:17.518807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:17.519431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:17.519746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:17.520018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:17.521689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:17.589261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:17.642938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:17.644377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325d1b8-bee3-4e32-adde-8af22c450938 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:17.664747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 668/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:20.763700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef219d-583c-4846-9cf5-9c7b1807ee53 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:20.764303 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef219d-583c-4846-9cf5-9c7b1807ee53 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:20.764511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef219d-583c-4846-9cf5-9c7b1807ee53 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:20.766278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 652/1320] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:20.780845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08410d-5682-41a5-9ed8-7dd94b3635d8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:20.781414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08410d-5682-41a5-9ed8-7dd94b3635d8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:20.781414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08410d-5682-41a5-9ed8-7dd94b3635d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:20.855477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9f08410d-5682-41a5-9ed8-7dd94b3635d8 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:20.856739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9f08410d-5682-41a5-9ed8-7dd94b3635d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:20.957188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 669/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:20.996131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef219d-583c-4846-9cf5-9c7b1807ee53 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:20.997772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2cef219d-583c-4846-9cf5-9c7b1807ee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:21.051482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:21.111914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef219d-583c-4846-9cf5-9c7b1807ee53 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:21.195079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:21.195729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:21.196042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:21.196282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:21.197942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:21.320387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:21.335707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:21.337173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c58be-d747-43c0-8647-720befb24039 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:21.357432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 653/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:24.270065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cecd29-77f9-4c69-b011-d9676a3cad20 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:24.270065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cecd29-77f9-4c69-b011-d9676a3cad20 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:24.270065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cecd29-77f9-4c69-b011-d9676a3cad20 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:24.271015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 670/1323] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:24.276723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ae5aa-ac45-4f40-8767-59d869ed32f4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:24.277256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ae5aa-ac45-4f40-8767-59d869ed32f4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:24.277256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ae5aa-ac45-4f40-8767-59d869ed32f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:24.351734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a44ae5aa-ac45-4f40-8767-59d869ed32f4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:24.352969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a44ae5aa-ac45-4f40-8767-59d869ed32f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:24.483265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 654/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:24.550803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cecd29-77f9-4c69-b011-d9676a3cad20 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:24.554371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-16cecd29-77f9-4c69-b011-d9676a3cad20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:24.584155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:24.737668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cecd29-77f9-4c69-b011-d9676a3cad20 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:24.849491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:24.850038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:24.850324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:24.850600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:24.852372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:24.947656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:25.001999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:25.003637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fbfe7-f378-4cd8-aa86-c6d1bda09ea2 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:25.028237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 671/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:27.220601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3ff4e-5797-4f99-ba4d-200b89f402cf None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.221394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3ff4e-5797-4f99-ba4d-200b89f402cf None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.221753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3ff4e-5797-4f99-ba4d-200b89f402cf None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.223477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 655/1326] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:27.229440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18787e63-9fe4-434b-8096-3cbf7915f788 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.230008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18787e63-9fe4-434b-8096-3cbf7915f788 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.230008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18787e63-9fe4-434b-8096-3cbf7915f788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.307129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-18787e63-9fe4-434b-8096-3cbf7915f788 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:27.308392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-18787e63-9fe4-434b-8096-3cbf7915f788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.420380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 672/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:27.462873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3ff4e-5797-4f99-ba4d-200b89f402cf None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.465555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c1e3ff4e-5797-4f99-ba4d-200b89f402cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.550558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:27.613358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3ff4e-5797-4f99-ba4d-200b89f402cf None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.701718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.702361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.702650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.703042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.705359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.835995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:27.835995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.837887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c506d-1be5-495f-9dc3-bbb82a62d81d service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.870891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 656/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:11:30.017751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdb078-b756-44ca-9089-a4f0a0ad37d7 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:30.018302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdb078-b756-44ca-9089-a4f0a0ad37d7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:30.018302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdb078-b756-44ca-9089-a4f0a0ad37d7 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:30.019720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 673/1329] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:11:30.025664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1bff7-2d36-4e77-bdd0-cf324829bd36 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:30.026228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1bff7-2d36-4e77-bdd0-cf324829bd36 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:30.026316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1bff7-2d36-4e77-bdd0-cf324829bd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:30.108959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-67b1bff7-2d36-4e77-bdd0-cf324829bd36 None None] MFA Rules not processed for user `b0b3f354b8b6415a9dda8dd637e5335e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:30.110787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-67b1bff7-2d36-4e77-bdd0-cf324829bd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:30.230841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 657/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:30.270901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdb078-b756-44ca-9089-a4f0a0ad37d7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:30.272442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9fcdb078-b756-44ca-9089-a4f0a0ad37d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:30.308968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:30.393159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdb078-b756-44ca-9089-a4f0a0ad37d7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:30.480398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:30.481040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:30.481319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:30.481594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:30.483297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:30.578148 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:30.619802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:30.621266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56a7df-c9f1-4a0e-a5ef-64a25dc60766 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:30.646023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 674/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:33.091400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e9cb6-2441-474a-9dff-ce94613d99e9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.091989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e9cb6-2441-474a-9dff-ce94613d99e9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.091989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e9cb6-2441-474a-9dff-ce94613d99e9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.093399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 658/1332] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:33.099336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7179f79-c3d3-4038-abf4-715670cfb322 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.099923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7179f79-c3d3-4038-abf4-715670cfb322 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.099923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7179f79-c3d3-4038-abf4-715670cfb322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.176978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b7179f79-c3d3-4038-abf4-715670cfb322 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:33.178243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b7179f79-c3d3-4038-abf4-715670cfb322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:33.295066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 675/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:33.333591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e9cb6-2441-474a-9dff-ce94613d99e9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.335145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-302e9cb6-2441-474a-9dff-ce94613d99e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:33.410682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:33.449017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e9cb6-2441-474a-9dff-ce94613d99e9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.531239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.531935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.532219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.532495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.534175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:33.665461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.666796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a70c-7271-4d1c-ad5d-62942eb02e37 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.678504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:33.689149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 659/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:35.833364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa77e5-b7cc-42ce-8ea4-8b2a5bad2d46 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:35.833907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa77e5-b7cc-42ce-8ea4-8b2a5bad2d46 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:35.833907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa77e5-b7cc-42ce-8ea4-8b2a5bad2d46 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:35.835271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 676/1335] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:35.840856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03789e21-9701-4b28-86ff-7f215690d4ed None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:35.841378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03789e21-9701-4b28-86ff-7f215690d4ed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:35.841489 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03789e21-9701-4b28-86ff-7f215690d4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:35.913648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-03789e21-9701-4b28-86ff-7f215690d4ed None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:35.914805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-03789e21-9701-4b28-86ff-7f215690d4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:36.025867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 660/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:36.066673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa77e5-b7cc-42ce-8ea4-8b2a5bad2d46 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:36.068253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f8aa77e5-b7cc-42ce-8ea4-8b2a5bad2d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:36.116033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:36.187089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa77e5-b7cc-42ce-8ea4-8b2a5bad2d46 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:36.273537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:36.274156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:36.274435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:36.274705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:36.276396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:36.371256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:36.406179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:36.409159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d493cf-9115-4541-a413-24cef433e599 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:36.439179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 677/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:39.001066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0cf6-88b2-4702-a5c7-82755e33ed4d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:39.001615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0cf6-88b2-4702-a5c7-82755e33ed4d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:39.001615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0cf6-88b2-4702-a5c7-82755e33ed4d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:39.003009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 661/1338] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:39.008820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4671f-5bdb-4fb8-8231-a67c3252de40 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:39.009355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4671f-5bdb-4fb8-8231-a67c3252de40 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:39.009355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4671f-5bdb-4fb8-8231-a67c3252de40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:39.082073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6fd4671f-5bdb-4fb8-8231-a67c3252de40 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:39.083325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6fd4671f-5bdb-4fb8-8231-a67c3252de40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:39.208582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 678/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:39.245718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0cf6-88b2-4702-a5c7-82755e33ed4d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:39.247323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5bbc0cf6-88b2-4702-a5c7-82755e33ed4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:39.340671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:39.373774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0cf6-88b2-4702-a5c7-82755e33ed4d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:39.473980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:39.474625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:39.474994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:39.475175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:39.476935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:39.615182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:39.616581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a447-1b8a-4415-9c56-fbca8f747d89 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:39.637209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 662/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:44.151379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca17a-16d3-4f4a-a295-6f3f46997b17 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:44.152365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca17a-16d3-4f4a-a295-6f3f46997b17 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:44.152365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca17a-16d3-4f4a-a295-6f3f46997b17 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:44.153717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 679/1341] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:11:44.163110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8c188-98a3-42b1-acb3-787481f8dece None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:44.164117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8c188-98a3-42b1-acb3-787481f8dece None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:44.164117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8c188-98a3-42b1-acb3-787481f8dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:44.174860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:44.286881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a9e8c188-98a3-42b1-acb3-787481f8dece None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:44.288841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a9e8c188-98a3-42b1-acb3-787481f8dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:44.442931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 663/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:44.488643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca17a-16d3-4f4a-a295-6f3f46997b17 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:44.490331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c11ca17a-16d3-4f4a-a295-6f3f46997b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:44.558342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:44.643242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca17a-16d3-4f4a-a295-6f3f46997b17 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:44.785509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:44.786503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:44.786962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:44.787430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:44.790677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:44.942881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:44.980412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:44.983303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199aac4-5b39-4588-97f1-7faac036be9b service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:45.021886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 680/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:48.178061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a940-9365-46af-82ff-98460942e6e6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:48.178885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a940-9365-46af-82ff-98460942e6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:48.178885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a940-9365-46af-82ff-98460942e6e6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:48.183516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 664/1344] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:48.190682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe8521-2676-42b9-8b7a-d0ad8c918237 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:48.191239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe8521-2676-42b9-8b7a-d0ad8c918237 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:48.191670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe8521-2676-42b9-8b7a-d0ad8c918237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:48.299026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-31fe8521-2676-42b9-8b7a-d0ad8c918237 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:48.300671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-31fe8521-2676-42b9-8b7a-d0ad8c918237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:48.430371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 681/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:48.471958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363a940-9365-46af-82ff-98460942e6e6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:48.473628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b363a940-9365-46af-82ff-98460942e6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:48.580889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:48.613908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363a940-9365-46af-82ff-98460942e6e6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:48.737655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:48.738604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:48.739102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:48.739633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:48.742941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:48.942460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:48.945370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744e935-272d-40b4-9a1c-df7110bed4f8 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:48.982902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 665/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:55.042733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6871-88ce-4b7a-81c2-655ef977a6e9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:55.043696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6871-88ce-4b7a-81c2-655ef977a6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:55.043696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6871-88ce-4b7a-81c2-655ef977a6e9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:55.045586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 682/1347] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:11:55.054341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7545085-3efe-4912-852e-eec7391d134e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:55.055198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7545085-3efe-4912-852e-eec7391d134e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:55.055198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7545085-3efe-4912-852e-eec7391d134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:55.073676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:55.146506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a7545085-3efe-4912-852e-eec7391d134e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:55.148322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a7545085-3efe-4912-852e-eec7391d134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:55.326852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 666/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:55.377652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a6871-88ce-4b7a-81c2-655ef977a6e9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:55.379442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a28a6871-88ce-4b7a-81c2-655ef977a6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:55.528439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:55.622108 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a6871-88ce-4b7a-81c2-655ef977a6e9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:55.761781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:55.762405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:55.762759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:55.763390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:55.764920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:55.950180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:55.956503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:55.958043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab0e1-5b7f-46d3-b95b-df8b9960a34d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:55.983763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 683/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:02.575089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7d64-25bd-4c31-928d-2733d64f67db None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:02.575916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7d64-25bd-4c31-928d-2733d64f67db None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:02.576146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7d64-25bd-4c31-928d-2733d64f67db None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:02.579985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 667/1350] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:12:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:02.599299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5e46e-dba3-4608-8f93-a548e2e88f59 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:02.599299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5e46e-dba3-4608-8f93-a548e2e88f59 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:02.599299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5e46e-dba3-4608-8f93-a548e2e88f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:02.832844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-95d5e46e-dba3-4608-8f93-a548e2e88f59 None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:02.834098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-95d5e46e-dba3-4608-8f93-a548e2e88f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:03.054673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 684/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:12:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:05.328821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=59 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:09.110016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e8626-799f-4d2f-bb5b-84aab20cea0b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:09.111507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e8626-799f-4d2f-bb5b-84aab20cea0b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:09.112001 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e8626-799f-4d2f-bb5b-84aab20cea0b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:09.114458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 668/1352] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:12:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:09.119911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a051-6b6b-4a7f-ba8c-bc57e52dfe20 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:09.120730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a051-6b6b-4a7f-ba8c-bc57e52dfe20 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:09.120730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a051-6b6b-4a7f-ba8c-bc57e52dfe20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:09.204174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:09.244524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c1f7a051-6b6b-4a7f-ba8c-bc57e52dfe20 None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:09.246594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c1f7a051-6b6b-4a7f-ba8c-bc57e52dfe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:09.482849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 685/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:10.985814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88bab0-c5d9-4e97-8273-62ee0d8a4864 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:10.986628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88bab0-c5d9-4e97-8273-62ee0d8a4864 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:10.986782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88bab0-c5d9-4e97-8273-62ee0d8a4864 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:10.989275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 669/1354] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:12:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:10.997470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72922-ba7d-4ea4-af26-ba0fe71210ac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:10.998449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72922-ba7d-4ea4-af26-ba0fe71210ac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:10.998449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72922-ba7d-4ea4-af26-ba0fe71210ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:11.119539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5da72922-ba7d-4ea4-af26-ba0fe71210ac None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:11.121609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5da72922-ba7d-4ea4-af26-ba0fe71210ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:11.191030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:11.286766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 686/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:12:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:15.444446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fa7cc-c5da-48e9-ab6e-de6046d484a4 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:15.445357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fa7cc-c5da-48e9-ab6e-de6046d484a4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:15.445357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fa7cc-c5da-48e9-ab6e-de6046d484a4 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:15.446217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 670/1356] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:15.452879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d6bc-6ec6-4c2c-a960-4ac9e601ec02 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:15.453569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d6bc-6ec6-4c2c-a960-4ac9e601ec02 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:15.453569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d6bc-6ec6-4c2c-a960-4ac9e601ec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:15.551886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3ae3d6bc-6ec6-4c2c-a960-4ac9e601ec02 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:15.554055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3ae3d6bc-6ec6-4c2c-a960-4ac9e601ec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:15.719896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 687/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:15.785656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654e474-1773-4eb5-80de-1f303944f3b6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:15.786585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654e474-1773-4eb5-80de-1f303944f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:15.787187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654e474-1773-4eb5-80de-1f303944f3b6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:15.790097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 671/1358] 10.222.0.17 () {54 vars in 962 bytes} [Mon Aug 1 14:12:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:15.802494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb27ce-1a25-4638-8280-09bb45f6c62f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:15.803227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb27ce-1a25-4638-8280-09bb45f6c62f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:15.803856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb27ce-1a25-4638-8280-09bb45f6c62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:15.837558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:15.924356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-71eb27ce-1a25-4638-8280-09bb45f6c62f None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:15.925802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-71eb27ce-1a25-4638-8280-09bb45f6c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:16.166327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 688/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Aug 1 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:16.191836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654e474-1773-4eb5-80de-1f303944f3b6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:16.201376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7654e474-1773-4eb5-80de-1f303944f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:16.507710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654e474-1773-4eb5-80de-1f303944f3b6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:16.643363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:16.709673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:16.710430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:16.710762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:16.711075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:16.714066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:16.908323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:16.911175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bb31a-d74e-48ab-9fab-2464386f05e1 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:16.948590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 672/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:20.257806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12213cb4-6657-437b-8e99-39405ae1b945 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:20.258516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12213cb4-6657-437b-8e99-39405ae1b945 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:20.258516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12213cb4-6657-437b-8e99-39405ae1b945 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:20.260079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 689/1361] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:20.266660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6afa16-078b-46c6-a96e-9b34bc41793e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:20.267206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6afa16-078b-46c6-a96e-9b34bc41793e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:20.267206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6afa16-078b-46c6-a96e-9b34bc41793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:20.352655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fe6afa16-078b-46c6-a96e-9b34bc41793e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:20.353908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fe6afa16-078b-46c6-a96e-9b34bc41793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:20.403385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:20.491725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 673/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:20.548557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12213cb4-6657-437b-8e99-39405ae1b945 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:20.550220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-12213cb4-6657-437b-8e99-39405ae1b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:20.689395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:20.879428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12213cb4-6657-437b-8e99-39405ae1b945 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:21.162579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:21.164212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:21.165042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:21.165832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:21.170007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:21.227683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:21.394898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:21.398450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4b6fd-ad5a-4bba-a1b4-c1263e3ef830 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:21.446438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 690/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 898 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:25.134121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07333b-9aae-440b-809d-29b3a6fb1a37 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:25.134913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07333b-9aae-440b-809d-29b3a6fb1a37 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:25.134913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07333b-9aae-440b-809d-29b3a6fb1a37 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:25.141361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 674/1364] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:25.149988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df814db-48e4-4e53-8400-577e9383406d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:25.150908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df814db-48e4-4e53-8400-577e9383406d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:25.150908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df814db-48e4-4e53-8400-577e9383406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:25.271406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7df814db-48e4-4e53-8400-577e9383406d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:25.275117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7df814db-48e4-4e53-8400-577e9383406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:25.489519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:25.499226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 691/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:25.556977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852d4b-d8b2-47cd-b8b7-61c54cb00768 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:25.557491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852d4b-d8b2-47cd-b8b7-61c54cb00768 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:25.558106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852d4b-d8b2-47cd-b8b7-61c54cb00768 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:25.560032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 675/1366] 10.222.0.17 () {54 vars in 962 bytes} [Mon Aug 1 14:12:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:25.576453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bdfbf-a481-4ace-ad00-e44cb23e0336 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:25.577716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bdfbf-a481-4ace-ad00-e44cb23e0336 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:25.578284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bdfbf-a481-4ace-ad00-e44cb23e0336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:25.775391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fc2bdfbf-a481-4ace-ad00-e44cb23e0336 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:25.777847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fc2bdfbf-a481-4ace-ad00-e44cb23e0336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:26.056045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 692/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Aug 1 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:26.066043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97852d4b-d8b2-47cd-b8b7-61c54cb00768 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:26.067899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-97852d4b-d8b2-47cd-b8b7-61c54cb00768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:26.362140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97852d4b-d8b2-47cd-b8b7-61c54cb00768 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:26.405246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:26.635741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:26.636740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:26.637233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:26.637730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:26.641851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:26.853538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:26.856851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a327f-4dd7-4664-b4b9-c730fda56074 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:26.896876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 676/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 831 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:28.293322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9117cc-179b-4775-abdc-c02a94e65de4 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.294100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9117cc-179b-4775-abdc-c02a94e65de4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.294272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9117cc-179b-4775-abdc-c02a94e65de4 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.298316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 693/1369] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:28.307304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57e02d-49dc-483d-8605-d1d18b9b727f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:28.308187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57e02d-49dc-483d-8605-d1d18b9b727f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:28.308624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57e02d-49dc-483d-8605-d1d18b9b727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:28.381935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:28.459750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1a57e02d-49dc-483d-8605-d1d18b9b727f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:28.461891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a57e02d-49dc-483d-8605-d1d18b9b727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:28.700011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 677/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:29.251432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e3968-53cf-4910-9b57-c58271402da3 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:29.252012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e3968-53cf-4910-9b57-c58271402da3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:29.252919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e3968-53cf-4910-9b57-c58271402da3 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:29.255884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 694/1371] 10.222.0.17 () {54 vars in 964 bytes} [Mon Aug 1 14:12:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:29.265745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c150e7-c19f-40df-8b7b-2b59a5d21528 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:29.266256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c150e7-c19f-40df-8b7b-2b59a5d21528 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:29.266744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c150e7-c19f-40df-8b7b-2b59a5d21528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:29.446055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-72c150e7-c19f-40df-8b7b-2b59a5d21528 None None] MFA Rules not processed for user `f3a7a9a0bda04d128a4f045fdb0bc80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:29.448604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-72c150e7-c19f-40df-8b7b-2b59a5d21528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:29.569181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:29.625686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 678/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Aug 1 14:12:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:29.634339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e3968-53cf-4910-9b57-c58271402da3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:29.635768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6e2e3968-53cf-4910-9b57-c58271402da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:29.739886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:29.757158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09799e37-0350-4899-b355-2b3efcb9d664 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:29.758543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09799e37-0350-4899-b355-2b3efcb9d664 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:29.759416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09799e37-0350-4899-b355-2b3efcb9d664 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:29.763885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 679/1373] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:29.780218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:29.780673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:29.781160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:29.859736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e3968-53cf-4910-9b57-c58271402da3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:29.930584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:29.932793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:30.044072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:30.044762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:30.045040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:30.045383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:30.047926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:30.182559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 680/1374] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:30.243447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:30.275214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:30.277165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:30.277547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:30.279371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0641c-d0ec-4f7b-8296-80d2c5d6865c service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:30.311978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 695/1375] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:30.408129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:30.575016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21543dc9-f47c-4137-9fab-c19b9ac8e1aa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:30.782805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:30.783449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:30.783766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:30.784045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:30.785861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:30.867018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:30.940931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:30.942662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5942c3-f308-4b70-a835-484ab1ae0be7 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:30.965220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 681/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:34.152768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8257e-dd78-4b6b-88a3-bb1e3b9ab313 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:34.153499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8257e-dd78-4b6b-88a3-bb1e3b9ab313 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:34.154007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8257e-dd78-4b6b-88a3-bb1e3b9ab313 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:34.157487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 696/1377] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:34 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:34.165953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035972ff-dae3-4b44-8350-f42ec594c517 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:34.166785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035972ff-dae3-4b44-8350-f42ec594c517 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:34.166785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035972ff-dae3-4b44-8350-f42ec594c517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:34.312368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-035972ff-dae3-4b44-8350-f42ec594c517 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:34.315260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-035972ff-dae3-4b44-8350-f42ec594c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:34.581902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 682/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:34.670897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8257e-dd78-4b6b-88a3-bb1e3b9ab313 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:34.673304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55f8257e-dd78-4b6b-88a3-bb1e3b9ab313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:34.948104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8257e-dd78-4b6b-88a3-bb1e3b9ab313 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:35.119113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:35.248272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:35.249809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:35.250276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:35.250835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:35.253403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:35.408598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:35.410809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ea8bb-a3dc-4c8e-ae45-ebe339c4cfef service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:35.437528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 697/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:12:37.764482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79deb19-3969-4c40-b7e2-1b72dce8c891 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:37.765298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79deb19-3969-4c40-b7e2-1b72dce8c891 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:37.765298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79deb19-3969-4c40-b7e2-1b72dce8c891 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:37.766796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 683/1380] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:37.775047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc1b38-e0fc-40d5-a91b-d1ba32a1c3c5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:37.775047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc1b38-e0fc-40d5-a91b-d1ba32a1c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:37.775840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc1b38-e0fc-40d5-a91b-d1ba32a1c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:37.872421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-10cc1b38-e0fc-40d5-a91b-d1ba32a1c3c5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:37.874991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-10cc1b38-e0fc-40d5-a91b-d1ba32a1c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:37.895800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:38.048672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 698/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:38.104318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79deb19-3969-4c40-b7e2-1b72dce8c891 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:38.105910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c79deb19-3969-4c40-b7e2-1b72dce8c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.137772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:38.347617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79deb19-3969-4c40-b7e2-1b72dce8c891 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:38.497400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:38.527679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:38.529198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:38.530645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:38.531704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:38.535401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:38.734742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:38.736953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d9588-e5c6-4fba-9b40-b29b63218678 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:38.766588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 684/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:41.323059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46ce00-0263-4afc-a690-a2cbb947a012 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:41.323796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46ce00-0263-4afc-a690-a2cbb947a012 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:41.324119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46ce00-0263-4afc-a690-a2cbb947a012 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:41.327319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 699/1383] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:41.336633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce361e99-c8c3-48bd-b292-d322914b479f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:41.337328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce361e99-c8c3-48bd-b292-d322914b479f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:41.337764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce361e99-c8c3-48bd-b292-d322914b479f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:41.451756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ce361e99-c8c3-48bd-b292-d322914b479f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:41.453640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ce361e99-c8c3-48bd-b292-d322914b479f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:41.543682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:41.620224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 685/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:41.684625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46ce00-0263-4afc-a690-a2cbb947a012 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:41.688372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ac46ce00-0263-4afc-a690-a2cbb947a012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:41.944888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:41.950161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46ce00-0263-4afc-a690-a2cbb947a012 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:42.150702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:42.151862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:42.152694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:42.153296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:42.157000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:42.335290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:42.339746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d471d9e-2954-42b7-93ed-5a69363487d9 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:42.372777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 700/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:44.798062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2bbd-62fe-4db9-9508-64462632f9ed None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:44.799186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2bbd-62fe-4db9-9508-64462632f9ed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:44.799186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2bbd-62fe-4db9-9508-64462632f9ed None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:44.800097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 686/1386] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:44.806655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bc067-7274-4a12-b9d6-fa551299490d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:44.807577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bc067-7274-4a12-b9d6-fa551299490d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:44.807746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bc067-7274-4a12-b9d6-fa551299490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:44.814083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:44.927832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d96bc067-7274-4a12-b9d6-fa551299490d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:44.929105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d96bc067-7274-4a12-b9d6-fa551299490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:45.106514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 701/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:45.160010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2bbd-62fe-4db9-9508-64462632f9ed None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:45.162119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5d2e2bbd-62fe-4db9-9508-64462632f9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:45.374713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2bbd-62fe-4db9-9508-64462632f9ed None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:45.440585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:45.577229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:45.578063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:45.578598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:45.578981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:45.581464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:45.745801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:45.747515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a1af2-73f0-49a9-805a-8048c0253dca service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:45.785496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 687/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:48.121401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d011-9315-4746-953a-c4844a4a51b7 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.122345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d011-9315-4746-953a-c4844a4a51b7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.122345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d011-9315-4746-953a-c4844a4a51b7 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.123209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 702/1389] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:48.130272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a648b3-3d35-41a1-ae35-a55d594f0ead None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.130986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a648b3-3d35-41a1-ae35-a55d594f0ead None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.131478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a648b3-3d35-41a1-ae35-a55d594f0ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.259379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:48.275160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a3a648b3-3d35-41a1-ae35-a55d594f0ead None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:48.276901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a3a648b3-3d35-41a1-ae35-a55d594f0ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:48.458028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 688/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:48.513987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d011-9315-4746-953a-c4844a4a51b7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:48.516047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-83e4d011-9315-4746-953a-c4844a4a51b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:48.633940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:48.767390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d011-9315-4746-953a-c4844a4a51b7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:48.838442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a781c-66a1-4c10-8c56-2f6cdf80187e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.838743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a781c-66a1-4c10-8c56-2f6cdf80187e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.839021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a781c-66a1-4c10-8c56-2f6cdf80187e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.840624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 689/1391] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:48.847210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc8f7a-8301-40f1-a4b5-d365cd8676b3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.847535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc8f7a-8301-40f1-a4b5-d365cd8676b3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.847841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc8f7a-8301-40f1-a4b5-d365cd8676b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.928825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d3bc8f7a-8301-40f1-a4b5-d365cd8676b3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:48.930218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d3bc8f7a-8301-40f1-a4b5-d365cd8676b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:48.989375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:48.990019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.990301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.990576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.992631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:49.025087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:49.058794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:49.067494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 690/1392] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:49.124328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:49.126062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee6c6b-73e7-4718-9469-192fa703d76e service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:49.156295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 703/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:12:49.157593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dbd51-bca4-4912-ae2e-a0559bb27c2b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:49.158338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dbd51-bca4-4912-ae2e-a0559bb27c2b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:49.158524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dbd51-bca4-4912-ae2e-a0559bb27c2b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:49.160232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 691/1394] 10.222.0.17 () {54 vars in 966 bytes} [Mon Aug 1 14:12:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:49.169330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a63e8-7857-4caf-9dd6-8ebaa340a022 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:49.169877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a63e8-7857-4caf-9dd6-8ebaa340a022 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:49.170322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a63e8-7857-4caf-9dd6-8ebaa340a022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:49.297993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-de5a63e8-7857-4caf-9dd6-8ebaa340a022 None None] MFA Rules not processed for user `ee96f24e85de4092b31081d44fe0a03d`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:49.299549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-de5a63e8-7857-4caf-9dd6-8ebaa340a022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:49.420482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 704/1395] 10.222.0.17 () {58 vars in 1063 bytes} [Mon Aug 1 14:12:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:49.429594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dbd51-bca4-4912-ae2e-a0559bb27c2b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:49.432621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-125dbd51-bca4-4912-ae2e-a0559bb27c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:49.578786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dbd51-bca4-4912-ae2e-a0559bb27c2b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:49.683057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:49.683917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:49.684356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:49.684852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:49.686914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:49.737904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:49.853097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:49.854733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1da53f-3d62-4090-87e3-9eb0c2b6ba5a service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:49.880249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 692/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:12:50.887904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fa265-1dbf-428d-992a-74d1193736a1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:50.889285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fa265-1dbf-428d-992a-74d1193736a1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:50.889285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fa265-1dbf-428d-992a-74d1193736a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:50.949649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:50.988617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fe5fa265-1dbf-428d-992a-74d1193736a1 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:50.990165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fe5fa265-1dbf-428d-992a-74d1193736a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:51.098672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 705/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:12:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:51.149921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4375c8-b1e5-473c-b682-7341cb8180ce None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:51.150475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4375c8-b1e5-473c-b682-7341cb8180ce None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:51.150475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4375c8-b1e5-473c-b682-7341cb8180ce None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:51.151932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 693/1398] 10.222.0.17 () {54 vars in 943 bytes} [Mon Aug 1 14:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:51.159123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb421-28a2-4923-b4fb-5c16a1e47ec2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:51.159647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb421-28a2-4923-b4fb-5c16a1e47ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:51.160112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb421-28a2-4923-b4fb-5c16a1e47ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:51.261722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c42bb421-28a2-4923-b4fb-5c16a1e47ec2 None None] MFA Rules not processed for user `3690b0c864cb48a4b55774bb8875b644`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:51.263032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c42bb421-28a2-4923-b4fb-5c16a1e47ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:51.348938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:51.406876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 706/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Mon Aug 1 14:12:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:51.416614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4375c8-b1e5-473c-b682-7341cb8180ce None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:51.418185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ad4375c8-b1e5-473c-b682-7341cb8180ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:51.440680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f24010-0290-4f18-9521-246443a7794e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:51.440961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f24010-0290-4f18-9521-246443a7794e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:51.441232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f24010-0290-4f18-9521-246443a7794e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:51.442677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 707/1400] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:51.450227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:51.450666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:51.451053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:51.543193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:51.545720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:51.615341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4375c8-b1e5-473c-b682-7341cb8180ce None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:51.619842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:51.689005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 708/1401] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:51.756776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:51.757415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:51.757700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:51.757977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:51.759928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:51.766757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:51.768503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:51.840406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:51.869604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:51.871938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a994af-a51f-4674-a0cc-2aa197fe49f6 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:51.884933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 694/1402] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:12:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:12:51.974230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdde68-de9b-44bd-a8a1-b05ed8095a6c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:52.176485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:52.177225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:52.177657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:52.178071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:52.180012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:52.250055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:52.384600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:52.387803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acdaa7-4483-4aff-ab05-d4c9c4bfd136 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:52.442208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 709/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:12:55.030722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a61e7f-ce76-4d64-a749-c5aec2f17e3c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:55.032008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a61e7f-ce76-4d64-a749-c5aec2f17e3c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:55.032008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a61e7f-ce76-4d64-a749-c5aec2f17e3c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:55.033304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 695/1404] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:12:55.041384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2b9d2-7ef7-4847-9b95-dc0dfd323cd1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:55.041934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2b9d2-7ef7-4847-9b95-dc0dfd323cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:55.041934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2b9d2-7ef7-4847-9b95-dc0dfd323cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:55.152111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-14a2b9d2-7ef7-4847-9b95-dc0dfd323cd1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:55.153518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-14a2b9d2-7ef7-4847-9b95-dc0dfd323cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:55.417817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 710/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:55.497404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a61e7f-ce76-4d64-a749-c5aec2f17e3c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:55.500217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-39a61e7f-ce76-4d64-a749-c5aec2f17e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:55.564694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:55.757427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a61e7f-ce76-4d64-a749-c5aec2f17e3c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:55.970826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:55.977310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:55.978978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:55.980363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:55.980884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:55.985450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:56.161959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:56.163697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f90a0c-c1e0-4af5-b2ef-16e9645decef service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:56.188020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 696/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:58.465785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fa87d-8170-4057-8992-089458b9fef1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:58.467156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fa87d-8170-4057-8992-089458b9fef1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:58.467156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fa87d-8170-4057-8992-089458b9fef1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:58.471836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 711/1407] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:12:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:12:58.475334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8515b-95cd-420b-98a7-edfbe91ae1ae None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:58.476220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8515b-95cd-420b-98a7-edfbe91ae1ae None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:58.476220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8515b-95cd-420b-98a7-edfbe91ae1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:58.606402 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e8a8515b-95cd-420b-98a7-edfbe91ae1ae None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:58.609318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e8a8515b-95cd-420b-98a7-edfbe91ae1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:58.732580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:58.805845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 697/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:58.869957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fa87d-8170-4057-8992-089458b9fef1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:58.871643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d59fa87d-8170-4057-8992-089458b9fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:58.882613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:59.114636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fa87d-8170-4057-8992-089458b9fef1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:59.246789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:59.315960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:59.316704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:59.317016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:59.317342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:59.319235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:59.467847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:59.469421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4a570-3515-4c70-af8c-25cf7b23c744 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:59.492155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 712/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:08.800276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=12 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:09.492656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=58 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:16.914752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e83b31-65e8-47b1-9c5f-21dcfafc1f27 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:16.915494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e83b31-65e8-47b1-9c5f-21dcfafc1f27 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:16.915494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e83b31-65e8-47b1-9c5f-21dcfafc1f27 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:16.916635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 698/1410] 10.222.0.17 () {54 vars in 918 bytes} [Mon Aug 1 14:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:16.923185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae43561-0295-431d-87d5-509f72f43d7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:16.923746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae43561-0295-431d-87d5-509f72f43d7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:16.923844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae43561-0295-431d-87d5-509f72f43d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:17.010484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5ae43561-0295-431d-87d5-509f72f43d7a None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:17.011797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5ae43561-0295-431d-87d5-509f72f43d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:17.130521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 713/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Aug 1 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:17.147878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95b16e-9423-4e99-9cc2-633921481410 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:17.148401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95b16e-9423-4e99-9cc2-633921481410 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:17.148899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95b16e-9423-4e99-9cc2-633921481410 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:17.151025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 699/1412] 10.222.0.17 () {54 vars in 918 bytes} [Mon Aug 1 14:13:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:17.157332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5375c4c-b056-47ff-87f2-fecba3ab7031 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:17.157717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5375c4c-b056-47ff-87f2-fecba3ab7031 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:17.158048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5375c4c-b056-47ff-87f2-fecba3ab7031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:17.243869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:17.247801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d5375c4c-b056-47ff-87f2-fecba3ab7031 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:17.249665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d5375c4c-b056-47ff-87f2-fecba3ab7031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:17.408936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 714/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Aug 1 14:13:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:20.021529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4517d-8b21-48f6-a904-c79fc42f5140 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:20.022139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4517d-8b21-48f6-a904-c79fc42f5140 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:20.022139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4517d-8b21-48f6-a904-c79fc42f5140 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:20.023548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:13:20.030276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926bfd7-7d69-446b-ac63-0a687a6533f9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:20.030738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926bfd7-7d69-446b-ac63-0a687a6533f9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:20.030856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926bfd7-7d69-446b-ac63-0a687a6533f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:20.098384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b926bfd7-7d69-446b-ac63-0a687a6533f9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:20.099639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b926bfd7-7d69-446b-ac63-0a687a6533f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:20.157261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:20.200793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:20.248201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4517d-8b21-48f6-a904-c79fc42f5140 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:20.250940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-18f4517d-8b21-48f6-a904-c79fc42f5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:20.415773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4517d-8b21-48f6-a904-c79fc42f5140 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:20.517424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:20.571540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:20.575506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:20.575849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:20.576841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:20.579112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:20.717853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:20.720926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a075cf8-f531-41d0-80e6-bc6fe3aeeffa service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:20.757906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 701/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:21.719227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315efe0-64a5-43bb-9eec-9c7229473697 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:21.719791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315efe0-64a5-43bb-9eec-9c7229473697 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:21.720167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315efe0-64a5-43bb-9eec-9c7229473697 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:21.721485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:21.726427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b920d15-2151-49ad-b2f4-3dc698e964d6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:21.726904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b920d15-2151-49ad-b2f4-3dc698e964d6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:21.727002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b920d15-2151-49ad-b2f4-3dc698e964d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:21.806399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8b920d15-2151-49ad-b2f4-3dc698e964d6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:21.807704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8b920d15-2151-49ad-b2f4-3dc698e964d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:21.835682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:21.908363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:22.061817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315efe0-64a5-43bb-9eec-9c7229473697 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:22.063370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4315efe0-64a5-43bb-9eec-9c7229473697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:22.235698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315efe0-64a5-43bb-9eec-9c7229473697 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:22.302493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:22.393094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:22.394254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:22.394631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:22.394936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:22.398032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:22.511435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:22.513564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866825f-07db-49b9-a7d7-2dfe53441900 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:22.536963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 717/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:23.487804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56059b06-c6b4-4618-be8a-255a56e0440a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:23.488384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56059b06-c6b4-4618-be8a-255a56e0440a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:23.488384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56059b06-c6b4-4618-be8a-255a56e0440a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:23.489856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:23.495281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171869d4-80f9-4f9e-85ea-89c976cd795f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:23.495826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171869d4-80f9-4f9e-85ea-89c976cd795f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:23.495939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171869d4-80f9-4f9e-85ea-89c976cd795f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:23.561459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:23.590004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-171869d4-80f9-4f9e-85ea-89c976cd795f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:23.591775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-171869d4-80f9-4f9e-85ea-89c976cd795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:23.703595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:23.878714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56059b06-c6b4-4618-be8a-255a56e0440a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:23.880467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-56059b06-c6b4-4618-be8a-255a56e0440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:24.068326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56059b06-c6b4-4618-be8a-255a56e0440a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:24.087040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:24.207335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:24.208072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:24.208394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:24.208763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:24.210556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:24.318127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:24.320071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a35d1-c07e-4135-bf2c-7c086678947a service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:24.345348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 704/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:25.371143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e31c1f-bdbf-4214-8935-dfa0c1242e51 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:25.371929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e31c1f-bdbf-4214-8935-dfa0c1242e51 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:25.372145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e31c1f-bdbf-4214-8935-dfa0c1242e51 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:25.373947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:25.380264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ab5fb-4ef9-441c-b957-fae1f15b3611 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:25.380983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ab5fb-4ef9-441c-b957-fae1f15b3611 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:25.380983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ab5fb-4ef9-441c-b957-fae1f15b3611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:25.404877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:25.482975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-997ab5fb-4ef9-441c-b957-fae1f15b3611 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:25.484255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-997ab5fb-4ef9-441c-b957-fae1f15b3611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:25.594635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:25.763791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e31c1f-bdbf-4214-8935-dfa0c1242e51 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:25.765358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b9e31c1f-bdbf-4214-8935-dfa0c1242e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:25.893024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:25.941748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e31c1f-bdbf-4214-8935-dfa0c1242e51 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:26.085135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:26.085824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:26.086154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:26.086467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:26.088380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:26.190875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:26.205721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:26.207418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f282a5f-98e4-42cf-8b21-6d85477682e5 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:26.230410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 720/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:13:27.224005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc53d51-e811-4ccd-8d2c-1e8752896c4d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:27.224928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc53d51-e811-4ccd-8d2c-1e8752896c4d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:27.224928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc53d51-e811-4ccd-8d2c-1e8752896c4d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:27.226778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:27.232769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931136-acaa-4cb9-a932-21671b1d545c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:27.233442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931136-acaa-4cb9-a932-21671b1d545c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:27.233442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931136-acaa-4cb9-a932-21671b1d545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:27.328012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-dc931136-acaa-4cb9-a932-21671b1d545c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:27.329449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-dc931136-acaa-4cb9-a932-21671b1d545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:27.598088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:27.766973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc53d51-e811-4ccd-8d2c-1e8752896c4d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:27.768539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0bc53d51-e811-4ccd-8d2c-1e8752896c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:27.870174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:27.943908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc53d51-e811-4ccd-8d2c-1e8752896c4d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:28.135156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:28.136340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:28.137535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:28.138017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:28.141275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:28.336386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:28.396662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:28.398298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefd6d0-25a7-41d6-9ba5-3b643b110dc3 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:28.420274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 707/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:13:29.304007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f8396-6704-40a4-94f9-5c694aaad6f8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:29.304922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f8396-6704-40a4-94f9-5c694aaad6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:29.304922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f8396-6704-40a4-94f9-5c694aaad6f8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:29.307450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:29.314361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38297a4-3bf8-4ba4-bc0e-b7ffe4583552 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:29.314688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38297a4-3bf8-4ba4-bc0e-b7ffe4583552 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:29.314975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38297a4-3bf8-4ba4-bc0e-b7ffe4583552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:29.410389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a38297a4-3bf8-4ba4-bc0e-b7ffe4583552 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:29.411735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a38297a4-3bf8-4ba4-bc0e-b7ffe4583552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:29.536199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:29.700197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f8396-6704-40a4-94f9-5c694aaad6f8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:29.701806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-143f8396-6704-40a4-94f9-5c694aaad6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:29.760141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:29.888317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f8396-6704-40a4-94f9-5c694aaad6f8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:30.026760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:30.028105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:30.028390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:30.028659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:30.030384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:30.038429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:30.141391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:30.143043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4b126-1d4e-40aa-a76f-8467aa553f02 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:30.164157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 723/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:31.016025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee085501-a97b-4c21-a082-0c86307b947b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:31.016625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee085501-a97b-4c21-a082-0c86307b947b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:31.016625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee085501-a97b-4c21-a082-0c86307b947b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:31.018041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:31.023639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c20a-ba0a-4c56-8fa8-00466155d5a0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:31.024135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c20a-ba0a-4c56-8fa8-00466155d5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:31.024241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c20a-ba0a-4c56-8fa8-00466155d5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:31.111733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4120c20a-ba0a-4c56-8fa8-00466155d5a0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:31.113122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4120c20a-ba0a-4c56-8fa8-00466155d5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:31.228965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:31.288581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:31.482135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee085501-a97b-4c21-a082-0c86307b947b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:31.483920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ee085501-a97b-4c21-a082-0c86307b947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:31.503883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:31.650123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee085501-a97b-4c21-a082-0c86307b947b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:31.773608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:31.814569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:31.815633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:31.816110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:31.816578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:31.819802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:31.976029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:31.977861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ae612-3758-430e-865d-11fea52f18bb service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:32.001694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 710/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:32.940048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebabd6-ee8e-4d32-bd4e-45426401aa6e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:32.940909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebabd6-ee8e-4d32-bd4e-45426401aa6e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:32.940909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebabd6-ee8e-4d32-bd4e-45426401aa6e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:32.942134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:13:32.948792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908305-1172-4f01-9d82-7a954b990d97 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:32.950311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908305-1172-4f01-9d82-7a954b990d97 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:32.950311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908305-1172-4f01-9d82-7a954b990d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:33.057203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-93908305-1172-4f01-9d82-7a954b990d97 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:33.058635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-93908305-1172-4f01-9d82-7a954b990d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:33.117816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:33.207946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:33.381048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebabd6-ee8e-4d32-bd4e-45426401aa6e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:33.382651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f7ebabd6-ee8e-4d32-bd4e-45426401aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:33.564424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebabd6-ee8e-4d32-bd4e-45426401aa6e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:33.634339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:33.707255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:33.708051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:33.708334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:33.708609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:33.710302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:33.820372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:33.821898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cccb7d-aa7b-493e-bb5b-33d00fd74941 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:33.843770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 726/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:34.694592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64027-245e-467e-af03-20163ccc6332 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:34.695188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64027-245e-467e-af03-20163ccc6332 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:34.695188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64027-245e-467e-af03-20163ccc6332 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:34.697014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:34.703248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d0f8-e947-49b9-963a-b1d36d39d464 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:34.703973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d0f8-e947-49b9-963a-b1d36d39d464 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:34.703973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d0f8-e947-49b9-963a-b1d36d39d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:34.798613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3219d0f8-e947-49b9-963a-b1d36d39d464 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:34.799898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3219d0f8-e947-49b9-963a-b1d36d39d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:34.802223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:34.922248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:35.083007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64027-245e-467e-af03-20163ccc6332 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:35.084842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-03b64027-245e-467e-af03-20163ccc6332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:35.262110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64027-245e-467e-af03-20163ccc6332 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:35.297117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:35.424476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:35.425133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:35.425423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:35.425689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:35.427604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:35.550686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:35.552335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210b200-da99-43b0-ae6c-e14736a59c19 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:35.575389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 713/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:37.602040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e48df-9bf4-4855-8b06-3914e9d645ba None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:37.602643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e48df-9bf4-4855-8b06-3914e9d645ba None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:37.602643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e48df-9bf4-4855-8b06-3914e9d645ba None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:37.604009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:37.610398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d039e-02ba-48e0-9a2a-604923fe6f78 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:37.611179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d039e-02ba-48e0-9a2a-604923fe6f78 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:37.611179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d039e-02ba-48e0-9a2a-604923fe6f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:37.661780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:37.696753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4a4d039e-02ba-48e0-9a2a-604923fe6f78 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:37.698092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4a4d039e-02ba-48e0-9a2a-604923fe6f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:37.809871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:37.851620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e48df-9bf4-4855-8b06-3914e9d645ba None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:37.853419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8f0e48df-9bf4-4855-8b06-3914e9d645ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:37.970241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e48df-9bf4-4855-8b06-3914e9d645ba None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:38.005950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:38.053979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:38.054670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:38.055043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:38.055370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:38.057150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:38.183016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:38.185007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00dd3d-231f-45dc-ab8b-56aec4fcd12a service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:38.204838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 729/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:40.556116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd50774-b32e-4c4c-a248-108202fda0aa None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:40.556711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd50774-b32e-4c4c-a248-108202fda0aa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:40.556820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd50774-b32e-4c4c-a248-108202fda0aa None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:40.559914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:40.569260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052db39-dd3b-438b-898a-da70c25ee342 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:40.569826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052db39-dd3b-438b-898a-da70c25ee342 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:40.570135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052db39-dd3b-438b-898a-da70c25ee342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:40.663276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7052db39-dd3b-438b-898a-da70c25ee342 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:40.665560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7052db39-dd3b-438b-898a-da70c25ee342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:40.689705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:40.796178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:40.859824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd50774-b32e-4c4c-a248-108202fda0aa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:40.861433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dbd50774-b32e-4c4c-a248-108202fda0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:40.993009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd50774-b32e-4c4c-a248-108202fda0aa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:41.010362 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:41.083128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:41.083798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:41.084083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:41.084359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:41.086064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:41.220983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:41.222454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d03f12-871c-4aae-a9d3-64ed26e01788 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:41.246376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 716/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:13:43.746594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae589c46-fc9c-4f01-ba05-dd10b3b831e0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:43.747296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae589c46-fc9c-4f01-ba05-dd10b3b831e0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:43.747296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae589c46-fc9c-4f01-ba05-dd10b3b831e0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:43.748525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:43.754383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca582-e3e7-472e-aeef-ebd9178d46aa None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:43.754899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca582-e3e7-472e-aeef-ebd9178d46aa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:43.755202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca582-e3e7-472e-aeef-ebd9178d46aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:43.810189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:43.830174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d2aca582-e3e7-472e-aeef-ebd9178d46aa None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:43.831416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d2aca582-e3e7-472e-aeef-ebd9178d46aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:43.965626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:44.006226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae589c46-fc9c-4f01-ba05-dd10b3b831e0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:44.007949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ae589c46-fc9c-4f01-ba05-dd10b3b831e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:44.127177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae589c46-fc9c-4f01-ba05-dd10b3b831e0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:44.178446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:44.223427 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:44.224471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:44.224975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:44.225485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:44.228828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:44.409062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:44.414471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797ac2c-8f42-4d02-8c29-af2f1c03457d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:44.436779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 732/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:46.707753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed360-9e52-4c04-9886-764f26354001 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:46.708591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed360-9e52-4c04-9886-764f26354001 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:46.708591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed360-9e52-4c04-9886-764f26354001 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:46.710821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:46.720213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c8f87-43cc-4266-a5d5-cdf9336dc021 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:46.720768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c8f87-43cc-4266-a5d5-cdf9336dc021 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:46.720768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c8f87-43cc-4266-a5d5-cdf9336dc021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:46.794109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-358c8f87-43cc-4266-a5d5-cdf9336dc021 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:46.795332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-358c8f87-43cc-4266-a5d5-cdf9336dc021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:46.817570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:46.912896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:46.952800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed360-9e52-4c04-9886-764f26354001 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:46.954541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-55fed360-9e52-4c04-9886-764f26354001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:47.083981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed360-9e52-4c04-9886-764f26354001 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:47.112711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:47.198945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:47.200096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:47.200489 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:47.200902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:47.203183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:47.351906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:47.354743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d78bd-5061-42bd-8c2b-97dd2ca70796 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:47.376134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 719/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:49.698093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b51e3-9d0a-46cd-8b87-e16d42e9321e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:49.699271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b51e3-9d0a-46cd-8b87-e16d42e9321e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:49.699271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b51e3-9d0a-46cd-8b87-e16d42e9321e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:49.700945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:49.707125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9f4d1-af20-461e-83bb-c15468bd5726 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:49.707755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9f4d1-af20-461e-83bb-c15468bd5726 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:49.707755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9f4d1-af20-461e-83bb-c15468bd5726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:49.788512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:49.792611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f3a9f4d1-af20-461e-83bb-c15468bd5726 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:49.794895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f3a9f4d1-af20-461e-83bb-c15468bd5726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:49.914584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:13:49.957562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b51e3-9d0a-46cd-8b87-e16d42e9321e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:49.959115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-313b51e3-9d0a-46cd-8b87-e16d42e9321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:50.085355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b51e3-9d0a-46cd-8b87-e16d42e9321e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:50.140277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:50.170487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:50.171028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:50.171309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:50.171615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:50.173323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:50.298403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:50.299984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380c4e6-ee17-4814-a0e4-719fc1c50ad9 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:50.320212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 735/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:52.480389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11173eb8-823c-4e9b-b029-bb76982ebe2d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:52.481037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11173eb8-823c-4e9b-b029-bb76982ebe2d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:52.481037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11173eb8-823c-4e9b-b029-bb76982ebe2d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:52.482319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:52.489186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ac5b-3237-4056-8d1b-fadd12ff9bc2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:52.489849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ac5b-3237-4056-8d1b-fadd12ff9bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:52.489849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ac5b-3237-4056-8d1b-fadd12ff9bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:52.624333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-58f7ac5b-3237-4056-8d1b-fadd12ff9bc2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:52.626666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-58f7ac5b-3237-4056-8d1b-fadd12ff9bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:52.667952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:52.753218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:52.792442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11173eb8-823c-4e9b-b029-bb76982ebe2d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:52.794016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-11173eb8-823c-4e9b-b029-bb76982ebe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:52.929810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11173eb8-823c-4e9b-b029-bb76982ebe2d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:52.993126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:53.051723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:53.052355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:53.052646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:53.052920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:53.054630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:53.181275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:53.183759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7925a0-5e2b-4cd1-aaef-a19500f1b356 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:53.210197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 722/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:55.521733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2faeac-9f5c-4b3c-9be7-190dbd7923b8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:55.522300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2faeac-9f5c-4b3c-9be7-190dbd7923b8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:55.522300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2faeac-9f5c-4b3c-9be7-190dbd7923b8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:55.523673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:55.531332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceddf9-9353-408d-8413-0d6abfcd58d1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:55.531913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceddf9-9353-408d-8413-0d6abfcd58d1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:55.532225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceddf9-9353-408d-8413-0d6abfcd58d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:55.640786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-78ceddf9-9353-408d-8413-0d6abfcd58d1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:55.642222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-78ceddf9-9353-408d-8413-0d6abfcd58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:55.653059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:55.765437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:55.810775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2faeac-9f5c-4b3c-9be7-190dbd7923b8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:55.812577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1a2faeac-9f5c-4b3c-9be7-190dbd7923b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:55.951977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2faeac-9f5c-4b3c-9be7-190dbd7923b8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:56.024551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:56.046808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:56.047443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:56.047754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:56.048030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:56.049786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:56.180417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:56.182019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d100c-a731-411a-b889-8afccc65e63d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:56.205803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 738/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:58.525282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ea2e6-d16a-41d8-abb5-162643711098 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:58.525904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ea2e6-d16a-41d8-abb5-162643711098 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:58.525904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ea2e6-d16a-41d8-abb5-162643711098 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:58.527998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:58.534815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae131c7a-1c76-4dbb-a9ec-62515b22eadb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:58.535417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae131c7a-1c76-4dbb-a9ec-62515b22eadb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:58.535664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae131c7a-1c76-4dbb-a9ec-62515b22eadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:58.623938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ae131c7a-1c76-4dbb-a9ec-62515b22eadb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:58.625332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ae131c7a-1c76-4dbb-a9ec-62515b22eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:58.676941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:58.747425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:58.789311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ea2e6-d16a-41d8-abb5-162643711098 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:58.791094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ec3ea2e6-d16a-41d8-abb5-162643711098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:58.935743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ea2e6-d16a-41d8-abb5-162643711098 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:58.992225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:59.040593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:59.041230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:59.041514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:59.042328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:59.044062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:59.173983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:59.176857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f85a4-e8f0-4195-908e-134c6988f108 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:59.207558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 725/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:01.682026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39f16-897d-417c-991c-e6f1b9ee56e3 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:01.683157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39f16-897d-417c-991c-e6f1b9ee56e3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:01.683640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39f16-897d-417c-991c-e6f1b9ee56e3 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:01.686210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:14:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:01.692975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ad9b-beb0-43c4-b443-8e689a8354f0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:01.693663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ad9b-beb0-43c4-b443-8e689a8354f0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:01.693663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ad9b-beb0-43c4-b443-8e689a8354f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:01.781917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0490ad9b-beb0-43c4-b443-8e689a8354f0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:01.784582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0490ad9b-beb0-43c4-b443-8e689a8354f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:01.809179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:01.912858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:01.951411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39f16-897d-417c-991c-e6f1b9ee56e3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:01.953136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-51e39f16-897d-417c-991c-e6f1b9ee56e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:02.080178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39f16-897d-417c-991c-e6f1b9ee56e3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:02.153936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:02.162060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:02.162689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:02.163034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:02.163244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:02.164979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:02.305214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:02.306675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde58e4-e7e5-473c-9840-a0fb4979c2b6 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:02.327348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 741/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:04.468410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66000-5eec-486a-9c20-3ea9876d53eb None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:04.468988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66000-5eec-486a-9c20-3ea9876d53eb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:04.468988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c66000-5eec-486a-9c20-3ea9876d53eb None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:04.470604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:04.476901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38e069-5207-4279-b894-eedc71b1da7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:04.477487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38e069-5207-4279-b894-eedc71b1da7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:04.477604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38e069-5207-4279-b894-eedc71b1da7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:04.589590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8c38e069-5207-4279-b894-eedc71b1da7a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:04.591265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8c38e069-5207-4279-b894-eedc71b1da7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:04.661452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:04.721912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:04.761328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c66000-5eec-486a-9c20-3ea9876d53eb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:04.762936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a2c66000-5eec-486a-9c20-3ea9876d53eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:04.902282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c66000-5eec-486a-9c20-3ea9876d53eb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:04.955982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:04.986264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:04.986895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:04.987181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:04.987577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:04.989245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:05.130384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:05.132038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9307b1-99b2-4fa2-b04a-2e4777d8b9b3 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:05.154277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 728/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:14.721532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=22 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:15.152268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=66 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:28.009130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2aa10-2b89-4f73-ae44-57c4979f4c75 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:28.009843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2aa10-2b89-4f73-ae44-57c4979f4c75 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:28.010103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2aa10-2b89-4f73-ae44-57c4979f4c75 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:28.012620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 743/1471] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:28.019902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32148516-e173-46a2-ac0b-8d3a33c3b008 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:28.020757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32148516-e173-46a2-ac0b-8d3a33c3b008 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:28.020757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32148516-e173-46a2-ac0b-8d3a33c3b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:28.121924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-32148516-e173-46a2-ac0b-8d3a33c3b008 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:28.124018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-32148516-e173-46a2-ac0b-8d3a33c3b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:28.322268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 729/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:14:28.451395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6bb30-1ace-42d0-a5a1-9be4cbed7ea7 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:28.451727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6bb30-1ace-42d0-a5a1-9be4cbed7ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:28.451978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6bb30-1ace-42d0-a5a1-9be4cbed7ea7 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:28.453423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 744/1473] 10.222.0.17 () {54 vars in 966 bytes} [Mon Aug 1 14:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:28.464806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371639aa-6a28-4bc5-ba99-0f3fd8b9a2e3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:28.465537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371639aa-6a28-4bc5-ba99-0f3fd8b9a2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:28.466796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371639aa-6a28-4bc5-ba99-0f3fd8b9a2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:28.636356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-371639aa-6a28-4bc5-ba99-0f3fd8b9a2e3 None None] MFA Rules not processed for user `ee96f24e85de4092b31081d44fe0a03d`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:28.638321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-371639aa-6a28-4bc5-ba99-0f3fd8b9a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:28.684456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:28.794063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 730/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Mon Aug 1 14:14:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:28.807825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6bb30-1ace-42d0-a5a1-9be4cbed7ea7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:28.810967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bbe6bb30-1ace-42d0-a5a1-9be4cbed7ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:28.989222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6bb30-1ace-42d0-a5a1-9be4cbed7ea7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:29.139026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:29.140356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:29.140747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:29.141199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:29.144407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:29.273858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:29.389818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:29.393889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4194b-bc8d-4989-9597-a403fdb8ee8f service glance] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:29.436016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 745/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:31.766256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae38f8a-5627-4c8e-8ebf-c78a85d2986d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:31.767200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae38f8a-5627-4c8e-8ebf-c78a85d2986d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:31.771068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae38f8a-5627-4c8e-8ebf-c78a85d2986d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:31.774214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 731/1476] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:31.785156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294390e-8ff8-4a86-8dcf-2691fd58be37 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:31.785784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294390e-8ff8-4a86-8dcf-2691fd58be37 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:31.786192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294390e-8ff8-4a86-8dcf-2691fd58be37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:31.866738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3294390e-8ff8-4a86-8dcf-2691fd58be37 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:31.868017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3294390e-8ff8-4a86-8dcf-2691fd58be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:31.997460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 746/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:32.023418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae38f8a-5627-4c8e-8ebf-c78a85d2986d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:32.026794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ae38f8a-5627-4c8e-8ebf-c78a85d2986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:32.201526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae38f8a-5627-4c8e-8ebf-c78a85d2986d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:32.262367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:32.333010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:32.334371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:32.334956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:32.335621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:32.337155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:32.339647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:14:32.339647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:14:32.339647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:14:32.340803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166d8b7-f8c5-4e9e-bbcc-368b2bac2a6b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:32.372899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 732/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Aug 1 14:14:32 2022] GET /identity/v3/projects => generated 2553 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:14:34.702048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a139e-3209-4836-9aae-9d97868d2b38 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:34.702599 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a139e-3209-4836-9aae-9d97868d2b38 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:34.702793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a139e-3209-4836-9aae-9d97868d2b38 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:34.704325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 747/1479] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:34.710985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444f7bd-b515-4fa7-a956-c52f1fe9178e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:34.711765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444f7bd-b515-4fa7-a956-c52f1fe9178e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:34.711765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444f7bd-b515-4fa7-a956-c52f1fe9178e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:34.796501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8444f7bd-b515-4fa7-a956-c52f1fe9178e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:34.797775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8444f7bd-b515-4fa7-a956-c52f1fe9178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:34.923939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 733/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:34.975153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a139e-3209-4836-9aae-9d97868d2b38 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:34.979450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e18a139e-3209-4836-9aae-9d97868d2b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:34.990386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:35.173123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a139e-3209-4836-9aae-9d97868d2b38 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:35.283304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:35.352707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:35.353518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:35.353896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:35.354265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:35.356358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:35.481374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:35.483306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edee3f-8a25-4cc7-91e7-b5fe839faf95 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:35.506449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 748/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:14:37.836724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de84616-ce20-4e9b-bfc0-4a647cedee30 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:37.837851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de84616-ce20-4e9b-bfc0-4a647cedee30 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:37.838160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de84616-ce20-4e9b-bfc0-4a647cedee30 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:37.840173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 734/1482] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:37.848478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048463-5607-405d-b62d-e4a754072ac7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:37.849065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048463-5607-405d-b62d-e4a754072ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:37.852603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048463-5607-405d-b62d-e4a754072ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:37.939561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-df048463-5607-405d-b62d-e4a754072ac7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:37.940743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-df048463-5607-405d-b62d-e4a754072ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:37.998585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:38.063610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 749/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:38.105458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de84616-ce20-4e9b-bfc0-4a647cedee30 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:38.107184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6de84616-ce20-4e9b-bfc0-4a647cedee30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:38.137535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:38.240660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de84616-ce20-4e9b-bfc0-4a647cedee30 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:38.339015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:38.339681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:38.339974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:38.340250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:38.342166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:38.437016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:38.482438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:38.483962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c944ee3-4f4f-47f5-b5ad-d6240b9c4733 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:38.506990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 735/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:40.858910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b99327-3434-4408-ba0a-eab9271186de None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:40.859547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b99327-3434-4408-ba0a-eab9271186de None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:40.859547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b99327-3434-4408-ba0a-eab9271186de None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:40.860965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 750/1485] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:40.867119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf43cbc-9e3c-46af-9ff7-505c0b4a0ba0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:40.867797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf43cbc-9e3c-46af-9ff7-505c0b4a0ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:40.867797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf43cbc-9e3c-46af-9ff7-505c0b4a0ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:40.950378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fbf43cbc-9e3c-46af-9ff7-505c0b4a0ba0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:40.951880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fbf43cbc-9e3c-46af-9ff7-505c0b4a0ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:41.081532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 736/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:41.124296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b99327-3434-4408-ba0a-eab9271186de None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:41.125944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-42b99327-3434-4408-ba0a-eab9271186de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:41.254334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b99327-3434-4408-ba0a-eab9271186de None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:41.342969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:41.344991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:41.345813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:41.346112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:41.346600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:41.348759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:41.479070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:41.480473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd590189-22b4-4155-9b06-e523fe14c8f0 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:41.500589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 751/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:43.789803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd461-8b0b-4fc4-995d-289429633278 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:43.791072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd461-8b0b-4fc4-995d-289429633278 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:43.791072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd461-8b0b-4fc4-995d-289429633278 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:43.792364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 737/1488] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:43.799616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afb7df-9a04-46c0-9eba-ffc6e87cb5a5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:43.800512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afb7df-9a04-46c0-9eba-ffc6e87cb5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:43.800710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afb7df-9a04-46c0-9eba-ffc6e87cb5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:43.929956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-18afb7df-9a04-46c0-9eba-ffc6e87cb5a5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:43.932490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-18afb7df-9a04-46c0-9eba-ffc6e87cb5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:44.004836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:44.075045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 752/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:44.124286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd461-8b0b-4fc4-995d-289429633278 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:44.125917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-84bfd461-8b0b-4fc4-995d-289429633278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:44.167167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:44.251634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd461-8b0b-4fc4-995d-289429633278 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:44.376931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:44.377956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:44.378567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:44.378962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:44.381826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:44.488815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:44.526283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:44.527791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8865-7e74-447a-a70c-407fe0cf918a service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:44.548989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 738/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:46.955101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906124e5-ac0c-45f0-9cc2-f77e148082a5 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:46.955729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906124e5-ac0c-45f0-9cc2-f77e148082a5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:46.955729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906124e5-ac0c-45f0-9cc2-f77e148082a5 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:46.957098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 753/1491] 10.222.0.17 () {54 vars in 926 bytes} [Mon Aug 1 14:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:46.963866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f78d-fd1f-4f9c-9173-2ae0ac14414a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:46.964504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f78d-fd1f-4f9c-9173-2ae0ac14414a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:46.964504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f78d-fd1f-4f9c-9173-2ae0ac14414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:47.044208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b7e5f78d-fd1f-4f9c-9173-2ae0ac14414a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:47.045390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b7e5f78d-fd1f-4f9c-9173-2ae0ac14414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:47.159292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 739/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Aug 1 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:14:47.202423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906124e5-ac0c-45f0-9cc2-f77e148082a5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:47.204350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-906124e5-ac0c-45f0-9cc2-f77e148082a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:47.325063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906124e5-ac0c-45f0-9cc2-f77e148082a5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:47.430052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:47.431250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:47.431565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:47.431842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:47.433612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:47.444749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:47.568765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:47.570211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabf6ad-fe7b-417a-b7c2-026337edd168 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:47.591968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 754/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:57.157135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=82 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:57.589066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=50 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:18.525213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:18.525988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:18.525988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:18.626443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:18.627901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:18.761171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 740/1494] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:18.769691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:18.770240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:18.770240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:18.792076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:18.793372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:18.802645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 755/1495] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:18.808122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:18.821423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723af2d-99d2-4f1e-9d89-6ea6520e041a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:18.826391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:18.827026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:18.827311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:18.827622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:18.828342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:16:18.828650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:18.829692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:18.829692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:16:18.829692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:16:18.830586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:18.843922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 741/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:16:18.853342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:18.863173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ff848-acdb-464d-9b05-ff078d11593d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:18.868044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:18.868691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:18.868974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:18.869250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:18.869931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:18.871880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:18.871880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:16:18.871880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:16:18.873481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:18.934235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 756/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:16:18 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:18.939121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:18.953232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f8b59-3c29-4f09-85e0-658e10a33878 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:19.018008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:19.026649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:19.027298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.027617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.027944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.028713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:19.032203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:19.032203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:16:19.032203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:16:19.034610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:19.060634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 742/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:16:18 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:16:19.065560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:19.077616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42b08a-b083-4fa0-8eea-44d0f799f309 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:19.082932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:19.083627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.083914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.084194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.084909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:19.085943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:19.085943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:16:19.085943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:16:19.086844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:19.101258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 757/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:16:19.107412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:19.119744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b250e04-b21d-4e6b-97fb-d4ad40816908 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:19.124671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:19.125308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.125603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.125882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.126594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:19.127664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:19.127664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:16:19.127664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:16:19.128556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032a888-23b1-47d6-b52e-51b93cb7f014 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:19.143698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 743/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:16:19.148550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:19.158967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040405d7-d8fc-4aaf-9679-03846b4944f0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:19.164180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:19.164877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.165162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.165445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] PATH_INFO: `/v3/projects/5f3bf1f664f1418faa4b3958e779b855/users/5b6df8f59040453c9ee44166f936a894/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.183437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3bf1f664f1418faa4b3958e779b855, user_id=5b6df8f59040453c9ee44166f936a894, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:19.184867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:19.184867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:16:19.184867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:16:19.187151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857bdf5-83bb-4cf0-ae50-85dfe6e4d3a7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:19.204770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 758/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:16:19 2022] PUT /identity/v3/projects/5f3bf1f664f1418faa4b3958e779b855/users/5b6df8f59040453c9ee44166f936a894/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:19.218374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277e65-9572-43d5-b389-db68dc8ecf9a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.218668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277e65-9572-43d5-b389-db68dc8ecf9a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.218980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277e65-9572-43d5-b389-db68dc8ecf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.251015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5a277e65-9572-43d5-b389-db68dc8ecf9a None None] MFA Rules not processed for user `5b6df8f59040453c9ee44166f936a894`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:19.252283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5a277e65-9572-43d5-b389-db68dc8ecf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:19.332433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 744/1502] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:16:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:19.346698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29211-574b-46a0-9ba1-75e96f356edb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.346982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29211-574b-46a0-9ba1-75e96f356edb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.347260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29211-574b-46a0-9ba1-75e96f356edb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.348667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 759/1503] 10.222.0.17 () {54 vars in 964 bytes} [Mon Aug 1 14:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:16:19.354657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ac539-4d10-411c-8dc5-12b884901d61 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.354947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ac539-4d10-411c-8dc5-12b884901d61 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.355235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ac539-4d10-411c-8dc5-12b884901d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.422727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b69ac539-4d10-411c-8dc5-12b884901d61 None None] MFA Rules not processed for user `f3a7a9a0bda04d128a4f045fdb0bc80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:19.424018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b69ac539-4d10-411c-8dc5-12b884901d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:19.510425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 745/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Aug 1 14:16:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:19.519487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd29211-574b-46a0-9ba1-75e96f356edb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:19.534104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd29211-574b-46a0-9ba1-75e96f356edb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:19.538448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:19.539097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.539383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.539678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.549662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:19.551331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:19.562831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 760/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:16:19.613232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560cc5c-e104-4bcf-922f-f646048fcb79 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.613758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560cc5c-e104-4bcf-922f-f646048fcb79 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.613860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560cc5c-e104-4bcf-922f-f646048fcb79 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.615311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 746/1506] 10.222.0.17 () {52 vars in 830 bytes} [Mon Aug 1 14:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:16:19.619338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:19.715362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db668b2-3006-4894-b8d7-03f9ca25ede1 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:19.723275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:19.725244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.725762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.726305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] PATH_INFO: `/v3/users/5b6df8f59040453c9ee44166f936a894` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.728449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6df8f59040453c9ee44166f936a894)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:19.730396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:19.730396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:16:19.730396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:16:19.732971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:19.844980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6df8f59040453c9ee44166f936a894'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:19.845734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6df8f59040453c9ee44166f936a894'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:19.875678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6df8f59040453c9ee44166f936a894'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:19.887130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6df8f59040453c9ee44166f936a894'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:19.901538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:19.907974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6df8f59040453c9ee44166f936a894'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:19.916702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 761/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:19 2022] DELETE /identity/v3/users/5b6df8f59040453c9ee44166f936a894 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:19.922087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:19.922413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:19.922637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:19.943050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:19.944343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:19.954372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:20.012311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 747/1508] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:20.023580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:20.025321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:20.111414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e3abe-09d0-4c10-8f60-31b8df9af66e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:20.115134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a2261-2187-4fcd-84e4-cf2011fb59d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:20.115803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2261-2187-4fcd-84e4-cf2011fb59d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:20.116088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2261-2187-4fcd-84e4-cf2011fb59d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:20.116367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2261-2187-4fcd-84e4-cf2011fb59d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:20.125974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a2261-2187-4fcd-84e4-cf2011fb59d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:20.127497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a2261-2187-4fcd-84e4-cf2011fb59d1 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:20.138570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 762/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:20.508912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:20.511782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:20.534763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b1d8b-14c4-4cff-bc98-73efeb47d67d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:20.542520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e39837-f391-461a-a952-092856e1566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:20.543197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39837-f391-461a-a952-092856e1566c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:20.543556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39837-f391-461a-a952-092856e1566c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:20.543754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e39837-f391-461a-a952-092856e1566c admin admin] PATH_INFO: `/v3/projects/5f3bf1f664f1418faa4b3958e779b855` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:20.545171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e39837-f391-461a-a952-092856e1566c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3bf1f664f1418faa4b3958e779b855)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:20.546306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-47e39837-f391-461a-a952-092856e1566c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:16:20.546306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:16:20.546306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:16:20.547635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e39837-f391-461a-a952-092856e1566c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:20.626422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47e39837-f391-461a-a952-092856e1566c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3bf1f664f1418faa4b3958e779b855', 'request_id': 'req-47e39837-f391-461a-a952-092856e1566c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:20.633899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47e39837-f391-461a-a952-092856e1566c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3bf1f664f1418faa4b3958e779b855', 'request_id': 'req-47e39837-f391-461a-a952-092856e1566c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:20.645856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47e39837-f391-461a-a952-092856e1566c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3bf1f664f1418faa4b3958e779b855', 'request_id': 'req-47e39837-f391-461a-a952-092856e1566c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:20.668071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47e39837-f391-461a-a952-092856e1566c admin admin] The token cache is being invalidate because project 5f3bf1f664f1418faa4b3958e779b855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:20.668366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47e39837-f391-461a-a952-092856e1566c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:20.669157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47e39837-f391-461a-a952-092856e1566c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:20.670722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 748/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:20 2022] DELETE /identity/v3/projects/5f3bf1f664f1418faa4b3958e779b855 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:30.134025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=38 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:30.664715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:05.617168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5041a-3660-4a78-a0c3-e7fba58ddda9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.617774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5041a-3660-4a78-a0c3-e7fba58ddda9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.617774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5041a-3660-4a78-a0c3-e7fba58ddda9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.619177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:05.624587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866893f-0e91-4b19-8ee2-f78c119fbb1e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:05.625375 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866893f-0e91-4b19-8ee2-f78c119fbb1e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:05.625375 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866893f-0e91-4b19-8ee2-f78c119fbb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:05.648847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d866893f-0e91-4b19-8ee2-f78c119fbb1e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:05.650770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d866893f-0e91-4b19-8ee2-f78c119fbb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:05.715517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:18:05.894890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5041a-3660-4a78-a0c3-e7fba58ddda9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:05.896865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-07a5041a-3660-4a78-a0c3-e7fba58ddda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.015739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5041a-3660-4a78-a0c3-e7fba58ddda9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.021291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d91dc-5651-4da6-a5e6-f09c96ac5ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.021952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d91dc-5651-4da6-a5e6-f09c96ac5ed4 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.022233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d91dc-5651-4da6-a5e6-f09c96ac5ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.022502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d91dc-5651-4da6-a5e6-f09c96ac5ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.031802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d91dc-5651-4da6-a5e6-f09c96ac5ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.033393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d91dc-5651-4da6-a5e6-f09c96ac5ed4 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.044777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 764/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:18:06.202932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38103e-37a9-4b5f-90ec-1b841f24672c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.203275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38103e-37a9-4b5f-90ec-1b841f24672c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.203639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38103e-37a9-4b5f-90ec-1b841f24672c None None] PATH_INFO: `/v3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.205031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 750/1514] 10.222.0.69 () {54 vars in 925 bytes} [Mon Aug 1 14:18:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:06.212632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fb80-2d90-47f8-9055-a0128b37f9d9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.212923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fb80-2d90-47f8-9055-a0128b37f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.213261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fb80-2d90-47f8-9055-a0128b37f9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.278926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9866fb80-2d90-47f8-9055-a0128b37f9d9 None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:06.280183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9866fb80-2d90-47f8-9055-a0128b37f9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.361889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 765/1515] 10.222.0.69 () {58 vars in 1015 bytes} [Mon Aug 1 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:06.920557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f8ff0-349a-4b52-84ac-9df39a88d2c2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.921124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f8ff0-349a-4b52-84ac-9df39a88d2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.921124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f8ff0-349a-4b52-84ac-9df39a88d2c2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.922462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 751/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:06.927238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f446f-a192-418a-b920-a843b501ad97 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.927693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f446f-a192-418a-b920-a843b501ad97 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.927840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f446f-a192-418a-b920-a843b501ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.948283 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2c2f446f-a192-418a-b920-a843b501ad97 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:06.949538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2c2f446f-a192-418a-b920-a843b501ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.962653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 766/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:07.115812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f8ff0-349a-4b52-84ac-9df39a88d2c2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.131250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f8ff0-349a-4b52-84ac-9df39a88d2c2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.136891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b0d91-7101-4bed-9c5a-64a47629a4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.137509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0d91-7101-4bed-9c5a-64a47629a4b1 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.137792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0d91-7101-4bed-9c5a-64a47629a4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.138070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0d91-7101-4bed-9c5a-64a47629a4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.146782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b0d91-7101-4bed-9c5a-64a47629a4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.148894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b0d91-7101-4bed-9c5a-64a47629a4b1 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.159380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 752/1518] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:15.295838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ffc9-aee3-4ea5-bedc-7f79ba267f80 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:15.296391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ffc9-aee3-4ea5-bedc-7f79ba267f80 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:15.296391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ffc9-aee3-4ea5-bedc-7f79ba267f80 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:15.297710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 767/1519] 10.222.0.17 () {54 vars in 961 bytes} [Mon Aug 1 14:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:15.302676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85a0d2-b0b5-4180-8b30-9a436425ad4d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:15.303083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85a0d2-b0b5-4180-8b30-9a436425ad4d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:15.303244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85a0d2-b0b5-4180-8b30-9a436425ad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:15.323508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3c85a0d2-b0b5-4180-8b30-9a436425ad4d None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:15.324738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3c85a0d2-b0b5-4180-8b30-9a436425ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:15.334280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 753/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Aug 1 14:18:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:15.342499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647ffc9-aee3-4ea5-bedc-7f79ba267f80 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:15.356918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647ffc9-aee3-4ea5-bedc-7f79ba267f80 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:15.362021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807aacc-11a0-48b8-983c-123579c6d139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:15.362642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807aacc-11a0-48b8-983c-123579c6d139 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:15.362921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807aacc-11a0-48b8-983c-123579c6d139 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:15.363199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807aacc-11a0-48b8-983c-123579c6d139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:15.372785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807aacc-11a0-48b8-983c-123579c6d139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:15.374396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807aacc-11a0-48b8-983c-123579c6d139 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:15.384933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 768/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:18.444488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae624d-d57a-4aa7-b925-af6a3a9d3c24 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.445100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae624d-d57a-4aa7-b925-af6a3a9d3c24 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.445100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae624d-d57a-4aa7-b925-af6a3a9d3c24 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.446611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:18.452873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6f7a2-8b94-44ba-b0ee-c8d04a1536f0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.453499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6f7a2-8b94-44ba-b0ee-c8d04a1536f0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.453499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6f7a2-8b94-44ba-b0ee-c8d04a1536f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.462639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:18.477815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-03d6f7a2-8b94-44ba-b0ee-c8d04a1536f0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:18.479677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-03d6f7a2-8b94-44ba-b0ee-c8d04a1536f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.487776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:18.656963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae624d-d57a-4aa7-b925-af6a3a9d3c24 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.672270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae624d-d57a-4aa7-b925-af6a3a9d3c24 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.678384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73a99e-8d93-4bc6-bfff-f21eeb27e924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.679014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a99e-8d93-4bc6-bfff-f21eeb27e924 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.679298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a99e-8d93-4bc6-bfff-f21eeb27e924 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.679612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a99e-8d93-4bc6-bfff-f21eeb27e924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.689232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73a99e-8d93-4bc6-bfff-f21eeb27e924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.690936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73a99e-8d93-4bc6-bfff-f21eeb27e924 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.703450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 755/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:19.541094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f84cf-876b-46f2-8d43-c161e399b5fb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.541722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f84cf-876b-46f2-8d43-c161e399b5fb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.541722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f84cf-876b-46f2-8d43-c161e399b5fb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.543031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:19.548863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cc5be-9a1f-474a-bfeb-9562e3ef7b9d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.549700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cc5be-9a1f-474a-bfeb-9562e3ef7b9d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.549700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cc5be-9a1f-474a-bfeb-9562e3ef7b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.570209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-252cc5be-9a1f-474a-bfeb-9562e3ef7b9d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:19.571449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-252cc5be-9a1f-474a-bfeb-9562e3ef7b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.581364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:19.783094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f84cf-876b-46f2-8d43-c161e399b5fb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.799742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f84cf-876b-46f2-8d43-c161e399b5fb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.806247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38a1de-ec80-4a76-a63a-962ca8692c85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.806927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38a1de-ec80-4a76-a63a-962ca8692c85 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.808004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38a1de-ec80-4a76-a63a-962ca8692c85 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.808423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38a1de-ec80-4a76-a63a-962ca8692c85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.818767 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38a1de-ec80-4a76-a63a-962ca8692c85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.820445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38a1de-ec80-4a76-a63a-962ca8692c85 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.831044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 771/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:24.080055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bafe8-2049-48d7-97ec-982e488e5ddb None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:24.080621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bafe8-2049-48d7-97ec-982e488e5ddb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:24.080621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bafe8-2049-48d7-97ec-982e488e5ddb None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:24.082018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:24.090726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b03f7-b0ee-4a0a-b310-29a82229583d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:24.091310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b03f7-b0ee-4a0a-b310-29a82229583d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:24.091310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b03f7-b0ee-4a0a-b310-29a82229583d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:24.112961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d99b03f7-b0ee-4a0a-b310-29a82229583d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:24.114228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d99b03f7-b0ee-4a0a-b310-29a82229583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:24.123300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:18:24.205775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bafe8-2049-48d7-97ec-982e488e5ddb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:24.207732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-404bafe8-2049-48d7-97ec-982e488e5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:24.270749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bafe8-2049-48d7-97ec-982e488e5ddb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:24.274409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fc146-c34d-456a-ba4c-c30028a65509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:24.275042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fc146-c34d-456a-ba4c-c30028a65509 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:24.275328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fc146-c34d-456a-ba4c-c30028a65509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:24.275669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fc146-c34d-456a-ba4c-c30028a65509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:24.285950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fc146-c34d-456a-ba4c-c30028a65509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:24.287531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fc146-c34d-456a-ba4c-c30028a65509 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:24.302832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 758/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:26.448414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28f4c4-fda7-45c0-bfff-3b99a39103e5 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:26.448940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28f4c4-fda7-45c0-bfff-3b99a39103e5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:26.448940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28f4c4-fda7-45c0-bfff-3b99a39103e5 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:26.450236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:26.456252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145bcb-09c9-4d40-a095-f9cab5cda99e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:26.456799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145bcb-09c9-4d40-a095-f9cab5cda99e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:26.456896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145bcb-09c9-4d40-a095-f9cab5cda99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:26.480677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-00145bcb-09c9-4d40-a095-f9cab5cda99e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:26.482280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-00145bcb-09c9-4d40-a095-f9cab5cda99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:26.490638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:18:26.528564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28f4c4-fda7-45c0-bfff-3b99a39103e5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:26.542479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28f4c4-fda7-45c0-bfff-3b99a39103e5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:26.546438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c9eb8-c441-4eac-9d4e-f3e3bb7193fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:26.547090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9eb8-c441-4eac-9d4e-f3e3bb7193fa service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:26.547371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9eb8-c441-4eac-9d4e-f3e3bb7193fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:26.547683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9eb8-c441-4eac-9d4e-f3e3bb7193fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:26.558990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c9eb8-c441-4eac-9d4e-f3e3bb7193fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:26.560606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c9eb8-c441-4eac-9d4e-f3e3bb7193fa service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:26.573074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 774/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:30.598430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5fde-c817-4feb-bb4e-f07762629499 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:30.599293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5fde-c817-4feb-bb4e-f07762629499 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:30.599531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b5fde-c817-4feb-bb4e-f07762629499 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:30.601185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:18:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:30.608707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859a05c-c892-4f7e-ba3e-3974a2773bf2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:30.609465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859a05c-c892-4f7e-ba3e-3974a2773bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:30.609721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859a05c-c892-4f7e-ba3e-3974a2773bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:30.631839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b859a05c-c892-4f7e-ba3e-3974a2773bf2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:30.633237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b859a05c-c892-4f7e-ba3e-3974a2773bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:30.649025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:30.995945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b5fde-c817-4feb-bb4e-f07762629499 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:30.997800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9b2b5fde-c817-4feb-bb4e-f07762629499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:31.005106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=75 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:31.071851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b5fde-c817-4feb-bb4e-f07762629499 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:31.076654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c2396-ecf9-4c93-a9f1-a24d5d170db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:31.077353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2396-ecf9-4c93-a9f1-a24d5d170db2 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:31.077678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2396-ecf9-4c93-a9f1-a24d5d170db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:31.077988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2396-ecf9-4c93-a9f1-a24d5d170db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:31.089001 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c2396-ecf9-4c93-a9f1-a24d5d170db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:31.090583 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c2396-ecf9-4c93-a9f1-a24d5d170db2 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:31.102322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 761/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:33.542731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d332f-7806-407e-8a3c-519717832879 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:33.543504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d332f-7806-407e-8a3c-519717832879 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:33.543504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d332f-7806-407e-8a3c-519717832879 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:33.544935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:33.551441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7d49d-1426-4547-87ef-de62a2033276 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:33.552342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7d49d-1426-4547-87ef-de62a2033276 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:33.552516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7d49d-1426-4547-87ef-de62a2033276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:33.576239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-dba7d49d-1426-4547-87ef-de62a2033276 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:33.577504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dba7d49d-1426-4547-87ef-de62a2033276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:33.587144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:18:33.941392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d332f-7806-407e-8a3c-519717832879 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:33.943308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4b5d332f-7806-407e-8a3c-519717832879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:33.958319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d332f-7806-407e-8a3c-519717832879 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:33.965056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702513c-a137-424b-835f-8722f2312ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:33.965707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702513c-a137-424b-835f-8722f2312ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:33.965990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702513c-a137-424b-835f-8722f2312ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:33.966269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702513c-a137-424b-835f-8722f2312ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:33.977124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702513c-a137-424b-835f-8722f2312ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:33.978627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702513c-a137-424b-835f-8722f2312ef6 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:33.989628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 777/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:43.568725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=28 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:43.984778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=26 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:07.121342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5df21-aaf3-4de6-9901-15439611bda4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.122641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5df21-aaf3-4de6-9901-15439611bda4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.123156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5df21-aaf3-4de6-9901-15439611bda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.142717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd76646-c266-42bf-b328-08a6dfde4c52 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.143310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd76646-c266-42bf-b328-08a6dfde4c52 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.143310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd76646-c266-42bf-b328-08a6dfde4c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.160289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2bd5df21-aaf3-4de6-9901-15439611bda4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.161937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2bd5df21-aaf3-4de6-9901-15439611bda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.182069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 763/1540] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.185587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1fd76646-c266-42bf-b328-08a6dfde4c52 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.187262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1fd76646-c266-42bf-b328-08a6dfde4c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.191441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baffa92-b28d-482f-a9bb-5ade6922b9a4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.192061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baffa92-b28d-482f-a9bb-5ade6922b9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.192565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baffa92-b28d-482f-a9bb-5ade6922b9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.208668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 778/1541] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:07.231482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.231482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.231482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.244539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7baffa92-b28d-482f-a9bb-5ade6922b9a4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.246182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7baffa92-b28d-482f-a9bb-5ade6922b9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.273074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.274839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.276468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 764/1542] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:07.277969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8b449-aab1-45bb-849b-6393aca6d7d1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.278490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8b449-aab1-45bb-849b-6393aca6d7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.279006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8b449-aab1-45bb-849b-6393aca6d7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.293416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 779/1543] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:07.294796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.326295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31980e9a-5246-4534-a9e1-7c7345427c88 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.327280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.332502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.332904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.333292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.333614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-82a8b449-aab1-45bb-849b-6393aca6d7d1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.334359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:07.334801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.335634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-82a8b449-aab1-45bb-849b-6393aca6d7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.336493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:20:07.336493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:20:07.336493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:20:07.339482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.358127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 765/1544] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.359876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce202-1cb0-4ed3-9002-16af1803db2a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.360392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce202-1cb0-4ed3-9002-16af1803db2a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.360888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce202-1cb0-4ed3-9002-16af1803db2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.372951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 780/1545] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:07.374740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.395673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9f9ce202-1cb0-4ed3-9002-16af1803db2a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.397342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9f9ce202-1cb0-4ed3-9002-16af1803db2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.399418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3380032-f88b-4eb1-8e05-55d4e25f0901 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.412423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 766/1546] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.413746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.414672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.419431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.420783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e97440-7823-4163-8938-095c9a64a8d5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.421293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.422286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:07.422811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e97440-7823-4163-8938-095c9a64a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.423350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.423861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e97440-7823-4163-8938-095c9a64a8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.426185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.457317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 781/1547] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:07.458522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.468281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-62e97440-7823-4163-8938-095c9a64a8d5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.469491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453a788-121b-4edc-be92-9cbd55bd3b6e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.469949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-62e97440-7823-4163-8938-095c9a64a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.475408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62589d47-52db-42c0-8046-a47189925083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.476439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62589d47-52db-42c0-8046-a47189925083 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.476933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62589d47-52db-42c0-8046-a47189925083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.477426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62589d47-52db-42c0-8046-a47189925083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.479327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62589d47-52db-42c0-8046-a47189925083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.481743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62589d47-52db-42c0-8046-a47189925083 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.481893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 767/1548] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.483725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60be63-6749-4945-833f-d826e2d2e4ac None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.484133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60be63-6749-4945-833f-d826e2d2e4ac None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.484520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60be63-6749-4945-833f-d826e2d2e4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.518378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6d60be63-6749-4945-833f-d826e2d2e4ac None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.522434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6d60be63-6749-4945-833f-d826e2d2e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.522618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 782/1549] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.522618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.522618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.522618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.543337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.544635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.613423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 768/1550] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.614849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.615134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.616013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.642697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.643795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 783/1551] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.644031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.648273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.662573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 769/1552] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.662573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.668058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170be4b-e7bf-4141-ab2f-8be65be88b9e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.675139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120b617-b683-4ecd-bd38-51e4443c4b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.675956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b617-b683-4ecd-bd38-51e4443c4b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.678841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b617-b683-4ecd-bd38-51e4443c4b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.678841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b617-b683-4ecd-bd38-51e4443c4b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.678841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120b617-b683-4ecd-bd38-51e4443c4b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.679276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ef33c-5064-4f92-8dd5-34b589981b60 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.680358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120b617-b683-4ecd-bd38-51e4443c4b97 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.697096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.700263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.700673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.701102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.702140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:07.702587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.705049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.716595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 784/1553] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.716595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8797b6-ed0c-481e-a0d7-a884eeba74a6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.716595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8797b6-ed0c-481e-a0d7-a884eeba74a6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.716595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8797b6-ed0c-481e-a0d7-a884eeba74a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.718541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 770/1554] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:07.719356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.734379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e4b4-d450-4fd1-94f9-6d5f7840e228 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.737232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9a8797b6-ed0c-481e-a0d7-a884eeba74a6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.738621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9a8797b6-ed0c-481e-a0d7-a884eeba74a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.797800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.798530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.798737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.798983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.799692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.801670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.806626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 785/1555] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.809017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99823f-d0a8-494c-be68-4102d3411397 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.809414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99823f-d0a8-494c-be68-4102d3411397 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.809829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99823f-d0a8-494c-be68-4102d3411397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.831395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 771/1556] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.832578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.840595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5c99823f-d0a8-494c-be68-4102d3411397 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.843413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5c99823f-d0a8-494c-be68-4102d3411397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.849049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44ad25-6f2a-40a2-bb2d-bec976f461d8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.856337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.857326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.857612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 786/1557] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.857874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.858257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.859380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:07.859566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.859925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.860051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.860411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.862825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8e14-1c1f-4e9f-b099-3cadbfa4dff4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.881453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 772/1558] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:07.882925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.883213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.883515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.893440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.894647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.903224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 787/1559] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.904227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.906070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:07.907593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:07.910863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:07.916288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386a9c0-5118-491f-b8d7-ed392128c566 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.918218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 773/1560] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:07.919007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:07.924780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.925425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.926145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.926337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.926891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.929008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:20:07.929008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:20:07.929008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:20:07.930540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f600f2-5854-4897-8e98-980f9c092d7a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.932626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd51eb7-aa77-425a-8742-b474d5bc7c94 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:07.940959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:07.941769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.942049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.942410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.943448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:07.944606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:20:07.944606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:20:07.944606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:20:07.945809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:07.967945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:07.976823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 788/1561] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/users => generated 289 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.979121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:07.979680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:07.980173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:07.986845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 774/1562] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:07.988360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.016203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:08.018026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:08.073223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d96c43-b311-48af-8d87-2aa9397d0ef9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.078036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.078772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.079185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.079590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.080399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.082145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.093882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 775/1563] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.094977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.107683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 789/1564] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:08.108546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.109069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a18db-27c7-447c-9e8e-a519063d52ba admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.117931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.118997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.119553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.120011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd320a-0d85-4c67-9ca1-1b9d5f2a38e8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.120186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.121374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:08.122086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.124779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c0563-c0ea-4b05-b0df-be02abff191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.124968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.125397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c0563-c0ea-4b05-b0df-be02abff191c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.125766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c0563-c0ea-4b05-b0df-be02abff191c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.126134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c0563-c0ea-4b05-b0df-be02abff191c admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.126840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c0563-c0ea-4b05-b0df-be02abff191c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.128826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c0563-c0ea-4b05-b0df-be02abff191c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.139796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 776/1565] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:08.140791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.153631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604990-7704-4f4a-b009-3a9fd0152950 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.160184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.160955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.161350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.161743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.163132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:08.163576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.165358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.165819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 790/1566] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.167086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.167383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.167686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.187670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 777/1567] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:08.187670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.189285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:08.190497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:08.191929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb474c-8328-4b12-8048-ce1a8db55180 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.256747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 791/1568] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:08.260535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.267295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd6902-b46c-4a81-8c94-7d30a4840c0b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.270859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.272333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.272551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.272765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.273172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.273271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.273454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.273732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.274275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.274395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.276429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.276770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.293035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 778/1569] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.294394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.303182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 792/1570] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/users => generated 284 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.303997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.311429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fedffb-7dce-410e-ae87-589ce88c10ba admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.314316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c605-9a2a-4f45-93d2-71abb184a79d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.316421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.317038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.317321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.317596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.318825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.319982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.320455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.320587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.320869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.321148 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.321852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:08.322164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.323954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.331545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 779/1571] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.332412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.341710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 793/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:08.342815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.343920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b56b-93e8-42a6-a45c-108fdc56638b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.351253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.352166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.352675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.352956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.353642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.355828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.357704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabfe8-3b9b-44d0-88fa-3814630799a0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.362428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.363043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.363324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.363627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.364320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.366313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.395526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 780/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.395526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.395947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 794/1574] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/users => generated 292 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.395947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.408067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3fa20-e339-4435-9657-ff5d010f321c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.421918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.429616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.468579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2db8-30fb-49b0-93b3-86a39706739e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.473974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.474147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.474637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.474761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.475002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.475105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.475202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.475296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.475920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:08.476034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.476232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.477825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.477986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.490817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 795/1575] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:08.492314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.502638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edabb61-aafc-4486-ba35-35d56b30e30a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.508345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.508989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.509271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.509548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.510253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.511921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.516993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 781/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/projects => generated 357 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.521052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.527823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 796/1577] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.527823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.535637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231e93-82b9-481a-9b9f-23be5a23328c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.594388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eef4cf-1f21-44fb-bce4-fc77ab6933b8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.599998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.600243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.600728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.600901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.601010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.601228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] PATH_INFO: `/v3/projects/a07d5eb233d349eab325368d9914f3e9/users/e47a94fa5e554608aadafdabadd901b9/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.601343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.601518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.602270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.603980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.614795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 782/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.616164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.616697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07d5eb233d349eab325368d9914f3e9, user_id=e47a94fa5e554608aadafdabadd901b9, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.620244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.628511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd127e-46fb-43e5-aea4-0109c1960be4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.638272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.639562 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.639753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 797/1579] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:08 2022] PUT /identity/v3/projects/a07d5eb233d349eab325368d9914f3e9/users/e47a94fa5e554608aadafdabadd901b9/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:08.639876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.640253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.640378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.641391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.644194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.676868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 783/1580] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/projects => generated 403 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.681245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.725045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da65bf7-02fe-4e45-b2c8-c510f7b56034 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.756794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cf1ff-e51d-40cd-ad30-fd92037347e7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.761677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.761867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.762305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.762588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.762906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.764094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.765895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.776365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 784/1581] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.777080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.787566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b798d-9589-468b-a068-50606e1adfc7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.792405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.793012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.793294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.793564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.794226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.796291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.812967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.814052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.814789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.815363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.816925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.821978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.829117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 785/1582] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.830276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.836747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.853661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 798/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.854484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.864564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8752a96-5a2d-4c15-a252-77b750643450 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.909838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b11b47-fc0f-4d0a-874b-6466bdd2e9ae admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.915985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.916790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.917181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.917553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.918449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.921420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.933219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5507dd-5224-4ed6-a3ae-9be2cdba3729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.934199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5507dd-5224-4ed6-a3ae-9be2cdba3729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.934540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5507dd-5224-4ed6-a3ae-9be2cdba3729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.934808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5507dd-5224-4ed6-a3ae-9be2cdba3729 admin admin] PATH_INFO: `/v3/projects/b9f5632d6b304db99e66877b0c329714/users/67fcd16d046c4df2a5dc99b43262375a/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.950418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 786/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/users => generated 279 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:08.951829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.957710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5507dd-5224-4ed6-a3ae-9be2cdba3729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f5632d6b304db99e66877b0c329714, user_id=67fcd16d046c4df2a5dc99b43262375a, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.960614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5507dd-5224-4ed6-a3ae-9be2cdba3729 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.966724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9123d4-012e-4ae8-819a-25e6daa3c890 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.972982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.973809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.974202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.974569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.975389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.976485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 799/1585] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:08 2022] PUT /identity/v3/projects/b9f5632d6b304db99e66877b0c329714/users/67fcd16d046c4df2a5dc99b43262375a/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:08.977531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.980988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.981430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.981839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.988424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 787/1586] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:08.989255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.000210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51e829-b773-484f-b321-832e39374f33 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.014959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] MFA Rules not processed for user `e47a94fa5e554608aadafdabadd901b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:09.016562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.073892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.074623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.074906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.075181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.075898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.078072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.100750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 800/1587] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:09.101609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.106692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 788/1588] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:08 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:09.106692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.115120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35adb-1113-4a01-8760-b379667c8c6a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.118722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c4b6-f549-4ddd-9192-29e0b4cf9e53 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.121403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.122046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.122330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.122614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.123813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.124948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.125315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.126446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.126909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.127334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.128570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.132200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.148140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 801/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.148140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.161456 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe384271-f4ec-48d5-a240-fd22f03755a6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.164945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 789/1590] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:09 2022] POST /identity/v3/users => generated 294 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:09.166512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.167590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ddfa4-7c3c-44a8-a6b3-6d6bab6d9817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.169010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ddfa4-7c3c-44a8-a6b3-6d6bab6d9817 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.169620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ddfa4-7c3c-44a8-a6b3-6d6bab6d9817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.169892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ddfa4-7c3c-44a8-a6b3-6d6bab6d9817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.170613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ddfa4-7c3c-44a8-a6b3-6d6bab6d9817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.172344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ddfa4-7c3c-44a8-a6b3-6d6bab6d9817 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.184125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f28d7a-2817-49b9-8ecd-898c8403dbeb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.184431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 802/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.185586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.185880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.186164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.191612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.192483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.192883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.193319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.194331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.196817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.207290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:09.208574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.219197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 790/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.219197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.219690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 803/1593] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:09.219690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.225082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:09.229838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b68d00-5d69-4485-abca-6fc58dae7461 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.235520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35506-74d1-474b-85fa-90c82dd2c873 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.238641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e943-401a-4da8-9f9b-335d553d9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.239499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e943-401a-4da8-9f9b-335d553d9cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.239869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e943-401a-4da8-9f9b-335d553d9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.240917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e943-401a-4da8-9f9b-335d553d9cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.241245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.241931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.242084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17e943-401a-4da8-9f9b-335d553d9cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.242233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.242482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] PATH_INFO: `/v3/projects/638b409b76a94100a7efeb925b86ea6f/users/f872c3471866499a9d9e2d2561f4a4e0/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.243741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17e943-401a-4da8-9f9b-335d553d9cc2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.258124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 804/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.259613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d19592-75f7-48c7-8b1f-06a498b85840 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.259912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d19592-75f7-48c7-8b1f-06a498b85840 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.260088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=638b409b76a94100a7efeb925b86ea6f, user_id=f872c3471866499a9d9e2d2561f4a4e0, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.260285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d19592-75f7-48c7-8b1f-06a498b85840 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.261128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:20:09.261128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:20:09.261128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:20:09.262232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 805/1595] 10.222.0.17 () {54 vars in 962 bytes} [Mon Aug 1 14:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:09.263029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d19592-75f7-48c7-8b1f-06a498b85840 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.264292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.280593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 791/1596] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:09 2022] PUT /identity/v3/projects/638b409b76a94100a7efeb925b86ea6f/users/f872c3471866499a9d9e2d2561f4a4e0/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:09.280915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d19592-75f7-48c7-8b1f-06a498b85840 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.281797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.317494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:09.358466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.359161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.359443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.359766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] PATH_INFO: `/v3/projects/3092982f3c6f4851a5a6e84bf7872d11/users/9eda4e19122444cf9cb2b6941adff9d5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.367285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c8732-1fe6-4e32-9b2b-d176217bf562 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.372924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.373890 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.374177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.374452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.375193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.376198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3092982f3c6f4851a5a6e84bf7872d11, user_id=9eda4e19122444cf9cb2b6941adff9d5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.376921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.378857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.388268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 792/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.389127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.393874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 806/1598] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:09 2022] PUT /identity/v3/projects/3092982f3c6f4851a5a6e84bf7872d11/users/9eda4e19122444cf9cb2b6941adff9d5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:09.395211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.400220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee7638-f61d-41f4-a196-c2e6b4d92347 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.476651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.477135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.477324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6da8c-b6c4-439a-b97a-6fff01f71e70 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.477491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.477684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.478556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.480567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.482689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.483326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.483824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.483985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.484739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.486285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.498062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 793/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.498062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.498571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 807/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.498571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.514788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7a5e-e776-4536-b025-f7d2066b0bcf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.525085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874af8c8-d6a3-403c-acaa-e9a79f0767e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.525085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874af8c8-d6a3-403c-acaa-e9a79f0767e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.525085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874af8c8-d6a3-403c-acaa-e9a79f0767e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.525085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874af8c8-d6a3-403c-acaa-e9a79f0767e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.526001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874af8c8-d6a3-403c-acaa-e9a79f0767e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.529131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874af8c8-d6a3-403c-acaa-e9a79f0767e3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.546309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 808/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.547996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.548323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.548604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.581941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c34db7-9672-432a-b5de-3629b13aa431 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.587027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01876ef0-fcb9-4e6a-8e4e-32f33167a213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.588132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01876ef0-fcb9-4e6a-8e4e-32f33167a213 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.588749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01876ef0-fcb9-4e6a-8e4e-32f33167a213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.589024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01876ef0-fcb9-4e6a-8e4e-32f33167a213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.593554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:09.595325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.603269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01876ef0-fcb9-4e6a-8e4e-32f33167a213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.604808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01876ef0-fcb9-4e6a-8e4e-32f33167a213 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.619987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 794/1602] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:09.621427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.621947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.622383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.661193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:09.678950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] MFA Rules not processed for user `f872c3471866499a9d9e2d2561f4a4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:09.681349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.698906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 809/1603] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Aug 1 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:09.707230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.719610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59e980-1e6c-430c-9b31-d3cb89a7f729 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.730128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfbbaf-35e5-49de-9278-9a28df814745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.730669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfbbaf-35e5-49de-9278-9a28df814745 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.730952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfbbaf-35e5-49de-9278-9a28df814745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.731215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfbbaf-35e5-49de-9278-9a28df814745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.731975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfbbaf-35e5-49de-9278-9a28df814745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.733645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfbbaf-35e5-49de-9278-9a28df814745 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.757400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 810/1604] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:09.759789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.760316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.760793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.761234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:09.787907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:09.789502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:09.796914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 795/1605] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:09.797771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.806986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 811/1606] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:09.809441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.822893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70005cdd-7350-41e5-8b01-c6c8dd325ece None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.824419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a5976-9492-4db2-b6ad-46c04e06edc7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:09.828806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.829846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.830024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:09.830360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.830638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:09.830828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] PATH_INFO: `/v3/projects/657664c470214f10bbada00c008ec87d/users/8b0c5b27345c44ea99d6e73e633f8fa3/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.830978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:09.831236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] PATH_INFO: `/v3/projects/24826d47eef7427083346628ec8de1d5/users/21d36a12c6424768a71eff3f151a6def/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:09.858039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] RBAC: Authorizing `identity:create_grant(project_id=657664c470214f10bbada00c008ec87d, user_id=8b0c5b27345c44ea99d6e73e633f8fa3, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.859886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24826d47eef7427083346628ec8de1d5, user_id=21d36a12c6424768a71eff3f151a6def, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:09.860978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.864026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:09.887815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 812/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:09 2022] PUT /identity/v3/projects/657664c470214f10bbada00c008ec87d/users/8b0c5b27345c44ea99d6e73e633f8fa3/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:09.889377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.894834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 796/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:09 2022] PUT /identity/v3/projects/24826d47eef7427083346628ec8de1d5/users/21d36a12c6424768a71eff3f151a6def/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:09.896034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:09.996483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca0817-ac4e-46c7-8828-b9d283e3544d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.023628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea2866-176b-40fc-aaa5-d61477ea3f5a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.031340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.032279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.032735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.033403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.080560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.081545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.081825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.082106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] PATH_INFO: `/v3/projects/26548c4abfb7420c97b621a1a8bfa196/users/ea7fd8ddcade4a279c59c8545eaa76d5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.104580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] RBAC: Authorizing `identity:create_grant(project_id=26548c4abfb7420c97b621a1a8bfa196, user_id=ea7fd8ddcade4a279c59c8545eaa76d5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.107593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.126922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 813/1609] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:09 2022] PUT /identity/v3/projects/26548c4abfb7420c97b621a1a8bfa196/users/ea7fd8ddcade4a279c59c8545eaa76d5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:10.128700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.138916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.145050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.159664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 797/1610] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:10.160615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.196485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:10.242352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8206c-064f-4d7f-9f27-c8895f74ce73 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.251989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:10.281978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a827c88-5988-481b-9799-e913a16983df admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.306753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.307581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.307936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.308191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] PATH_INFO: `/v3/projects/2fb1590048254549bb480d677892031f/users/fbaa67d901e34419b1616a8457fbd683/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.322338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c8bb-d158-47b1-a3c2-3d99ea804501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.322338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c8bb-d158-47b1-a3c2-3d99ea804501 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.322338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c8bb-d158-47b1-a3c2-3d99ea804501 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.322338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c8bb-d158-47b1-a3c2-3d99ea804501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.333386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb1590048254549bb480d677892031f, user_id=fbaa67d901e34419b1616a8457fbd683, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.336513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.357422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 814/1611] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:10 2022] PUT /identity/v3/projects/2fb1590048254549bb480d677892031f/users/fbaa67d901e34419b1616a8457fbd683/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:10.359012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.366203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.420781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c8bb-d158-47b1-a3c2-3d99ea804501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.422240 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c8bb-d158-47b1-a3c2-3d99ea804501 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.435679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 798/1612] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:10.437005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aae601-f78d-4d7b-9ec6-73634812d2f8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.437299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aae601-f78d-4d7b-9ec6-73634812d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.437578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aae601-f78d-4d7b-9ec6-73634812d2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.481122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c2aae601-f78d-4d7b-9ec6-73634812d2f8 None None] MFA Rules not processed for user `21d36a12c6424768a71eff3f151a6def`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.482496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c2aae601-f78d-4d7b-9ec6-73634812d2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.505339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc831b-f9f5-4f6b-ae01-a7d958be499d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.511961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579cb68-a8bb-4ee9-8bb2-4fc9b348f270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.512616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579cb68-a8bb-4ee9-8bb2-4fc9b348f270 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.512895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579cb68-a8bb-4ee9-8bb2-4fc9b348f270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.513168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579cb68-a8bb-4ee9-8bb2-4fc9b348f270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.648696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 799/1613] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.650232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fa61e-a7d6-4038-a067-78b444b1db32 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.650595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fa61e-a7d6-4038-a067-78b444b1db32 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.650885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fa61e-a7d6-4038-a067-78b444b1db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.674010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579cb68-a8bb-4ee9-8bb2-4fc9b348f270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.675508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579cb68-a8bb-4ee9-8bb2-4fc9b348f270 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.695450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 815/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:10.698537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef087-e656-4d57-9383-fffdd987cdcc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.698862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef087-e656-4d57-9383-fffdd987cdcc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.699316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef087-e656-4d57-9383-fffdd987cdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.711237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-524fa61e-a7d6-4038-a067-78b444b1db32 None None] MFA Rules not processed for user `8b0c5b27345c44ea99d6e73e633f8fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.715556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-524fa61e-a7d6-4038-a067-78b444b1db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.762772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1e0ef087-e656-4d57-9383-fffdd987cdcc None None] MFA Rules not processed for user `ea7fd8ddcade4a279c59c8545eaa76d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.764050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1e0ef087-e656-4d57-9383-fffdd987cdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.803746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:10.846453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 800/1615] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.846453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.846453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.846453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.853850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:10.871876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:10.873103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:10.882099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 801/1616] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.889984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.895656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 816/1617] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:10.900987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f1462-aaf0-4eba-b50f-74caf572c34e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.901519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f1462-aaf0-4eba-b50f-74caf572c34e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.901891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f1462-aaf0-4eba-b50f-74caf572c34e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.904798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 817/1618] 10.222.0.17 () {54 vars in 912 bytes} [Mon Aug 1 14:20:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:10.905501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f1462-aaf0-4eba-b50f-74caf572c34e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:10.911704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60b886-6ec7-458e-a3ad-aa53a0729b5e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.918568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.921861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.922223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.922634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.923376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.925445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.926458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f1462-aaf0-4eba-b50f-74caf572c34e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:10.940077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:10.940915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:10.941198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:10.941474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:10.942153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:10.944275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:10.966197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 802/1619] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/projects => generated 357 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:10.967737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.019021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 818/1620] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:10 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:11.020285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.070226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be3899-05f2-42e9-9181-9e480fed1996 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.150735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e579-c618-42ac-8f97-5271b53ae550 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.156227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8f1b-6440-4337-9da8-572735f28dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.157291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8f1b-6440-4337-9da8-572735f28dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.157633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8f1b-6440-4337-9da8-572735f28dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.157918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8f1b-6440-4337-9da8-572735f28dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.158602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc8f1b-6440-4337-9da8-572735f28dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.161329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc8f1b-6440-4337-9da8-572735f28dd8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.188451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.189551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.189840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.192010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.207768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 819/1621] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:11 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:11.207768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.207768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.207768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.215588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.217775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.230909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 803/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:11.231837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.266295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:11.267985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:11.269238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd077053-569f-4017-9ff7-f1a8ae69990d service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.432774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 820/1623] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:20:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:11.436355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.439020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.439745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.440034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.440532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.443753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.448804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.464066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658b296-6507-4f8d-9a5d-f5c2b287294a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.480902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.483224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.484246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.485748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.487150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.496161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.512957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 804/1624] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:11 2022] POST /identity/v3/users => generated 285 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:11.514677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.546195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:11.555847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 821/1625] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:11 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:11.557025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.563840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e235d87-40dc-49d9-bb3a-334a8907bb80 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.574387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.575147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.576430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.576808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.579445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.585884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.659025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 805/1626] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:11 2022] POST /identity/v3/users => generated 308 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:11.661409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.663976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:11.695009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a7968-4816-4d50-b2ab-323e92850dac admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.708219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.708905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.709181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.709463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.711881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.713518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.718197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcccdd-a7a8-46b0-8302-2028e5b3bf7f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.726984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.727663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.727950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.728227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.741428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 806/1627] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:11.741428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.754968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.757164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.764101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844729d0-01c7-4a87-9d2c-1ca027206ca1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.784430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.786074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.786668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.786793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 822/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:11.787294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.787685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.788855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.791821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.803857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c00ee-7743-4b67-8b59-43e513bf9c56 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.808973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.809987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.810475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.810720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 807/1629] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:11.811154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.811606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.812545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.815894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.842175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18465f8-da43-465d-bc23-c8899014dd6b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.843802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 823/1630] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:11.845044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.852869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.853661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.853943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.854246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] PATH_INFO: `/v3/projects/4c18ed7ee81a46e593d62422a96f3970/users/455c72bef439434dba9f5ff0d9e99181/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.866691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93737d85-1b26-4946-8862-8a1c3e1c8a9a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:11.873285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:11.874452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:11.875387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:11.876418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] PATH_INFO: `/v3/projects/9d3a6a77177a402896b08ae9595c19ae/users/0d0e57877eaa48c982972e0f098d4653/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:11.881767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c18ed7ee81a46e593d62422a96f3970, user_id=455c72bef439434dba9f5ff0d9e99181, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.885112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.914857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3a6a77177a402896b08ae9595c19ae, user_id=0d0e57877eaa48c982972e0f098d4653, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:11.918850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:11.926978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 808/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:11 2022] PUT /identity/v3/projects/4c18ed7ee81a46e593d62422a96f3970/users/455c72bef439434dba9f5ff0d9e99181/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:11.928122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:11.943943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 824/1632] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:11 2022] PUT /identity/v3/projects/9d3a6a77177a402896b08ae9595c19ae/users/0d0e57877eaa48c982972e0f098d4653/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:11.945878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:12.140331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9a799-345b-43c5-affd-6ba597aa9bdb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.158311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0252b-feb7-44ac-8c19-3150ccf198eb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.179888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.182121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.183138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.184957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.186431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.189472 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.213986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 825/1633] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:12.215229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:12.232578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d116af-e9d6-40d2-a676-a279a304bfff admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.240479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.241340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.242483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.242777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.243783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.248386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.273607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 826/1634] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:12.277953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:12.295350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.296616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.297255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.297798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] PATH_INFO: `/v3/projects/289056bc50f9476ca8d1445fbd61a622/users/7afa9896b553423b9d424a389b5f877e/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.327990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eb198-f102-4225-b207-b063592b8023 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.333261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289056bc50f9476ca8d1445fbd61a622, user_id=7afa9896b553423b9d424a389b5f877e, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.336840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.344218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.345590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.346781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.349716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] PATH_INFO: `/v3/projects/4c18ed7ee81a46e593d62422a96f3970/users/455c72bef439434dba9f5ff0d9e99181/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.355891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c18ed7ee81a46e593d62422a96f3970, user_id=455c72bef439434dba9f5ff0d9e99181, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.357824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 809/1635] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:11 2022] PUT /identity/v3/projects/289056bc50f9476ca8d1445fbd61a622/users/7afa9896b553423b9d424a389b5f877e/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:12.359505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:12.362191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.393691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 827/1636] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:12 2022] PUT /identity/v3/projects/4c18ed7ee81a46e593d62422a96f3970/users/455c72bef439434dba9f5ff0d9e99181/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:12.394521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:12.441187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:12.505151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44584bf-d268-436e-b962-4e416f7d1da1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.511778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3826-30f8-41c4-860a-a39b0ce2c316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.511975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f5b6d-bbfc-41ca-806c-ab15bbb80095 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.512941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3826-30f8-41c4-860a-a39b0ce2c316 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.513462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3826-30f8-41c4-860a-a39b0ce2c316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.513984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3826-30f8-41c4-860a-a39b0ce2c316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.515351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3826-30f8-41c4-860a-a39b0ce2c316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.518798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3826-30f8-41c4-860a-a39b0ce2c316 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.525201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.526339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.526622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.527410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] PATH_INFO: `/v3/projects/9d3a6a77177a402896b08ae9595c19ae/users/0d0e57877eaa48c982972e0f098d4653/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.537898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 828/1637] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:12.539788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00df4f-5795-46f3-9e69-fcf59947d0b5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.551909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00df4f-5795-46f3-9e69-fcf59947d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.553123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00df4f-5795-46f3-9e69-fcf59947d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.556698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3a6a77177a402896b08ae9595c19ae, user_id=0d0e57877eaa48c982972e0f098d4653, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.564559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:12.566392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.598413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ea00df4f-5795-46f3-9e69-fcf59947d0b5 None None] MFA Rules not processed for user `455c72bef439434dba9f5ff0d9e99181`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:12.598627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 810/1638] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:12 2022] PUT /identity/v3/projects/9d3a6a77177a402896b08ae9595c19ae/users/0d0e57877eaa48c982972e0f098d4653/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:12.600712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:12.609363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ea00df4f-5795-46f3-9e69-fcf59947d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:12.742627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 829/1639] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:12 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:12.745158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8cbb6-a009-448c-b5db-054c8a85f473 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.745706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8cbb6-a009-448c-b5db-054c8a85f473 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.746119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8cbb6-a009-448c-b5db-054c8a85f473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.797044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4ef8cbb6-a009-448c-b5db-054c8a85f473 None None] MFA Rules not processed for user `0d0e57877eaa48c982972e0f098d4653`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:12.804959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4ef8cbb6-a009-448c-b5db-054c8a85f473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:12.871969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636a76-badc-4b25-aa71-64b991bcd430 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:12.883403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:12.885233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:12.885527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:12.885802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] PATH_INFO: `/v3/projects/289056bc50f9476ca8d1445fbd61a622/users/7afa9896b553423b9d424a389b5f877e/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:12.891426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289056bc50f9476ca8d1445fbd61a622, user_id=7afa9896b553423b9d424a389b5f877e, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:12.898446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:12.938771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 811/1640] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:12 2022] PUT /identity/v3/projects/289056bc50f9476ca8d1445fbd61a622/users/7afa9896b553423b9d424a389b5f877e/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:12.938771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:13.005135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 830/1641] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:13.006688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.006840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.007074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.096721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] MFA Rules not processed for user `7afa9896b553423b9d424a389b5f877e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:13.098237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:13.119142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26260e4a-f1c0-4f5a-82c0-db5c89ff7963 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:13.124506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506a3e4-bd6f-4648-a803-6bfa3b6ce966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:13.125144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a3e4-bd6f-4648-a803-6bfa3b6ce966 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.125424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a3e4-bd6f-4648-a803-6bfa3b6ce966 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.125699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a3e4-bd6f-4648-a803-6bfa3b6ce966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.269755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:13.286554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 831/1642] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:13.288978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:13.294308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506a3e4-bd6f-4648-a803-6bfa3b6ce966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:13.300209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506a3e4-bd6f-4648-a803-6bfa3b6ce966 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:13.313595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3097c49-b597-4bf8-a57d-43028955c6ec None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:13.327004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 812/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:12 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:13.327877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:13.328348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c342c-bf35-45eb-b0b3-c07eea664a76 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.328538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.328640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c342c-bf35-45eb-b0b3-c07eea664a76 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.328812 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.328936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c342c-bf35-45eb-b0b3-c07eea664a76 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.329107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.330168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 813/1644] 10.222.0.17 () {54 vars in 962 bytes} [Mon Aug 1 14:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:13.338934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064779-7ec9-4496-9df2-7cb267d9cdd7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.339269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064779-7ec9-4496-9df2-7cb267d9cdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.339517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064779-7ec9-4496-9df2-7cb267d9cdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.384965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-eb064779-7ec9-4496-9df2-7cb267d9cdd7 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:13.386214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eb064779-7ec9-4496-9df2-7cb267d9cdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:13.403411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:13.491180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:13.492244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:13.519200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 832/1645] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:13.521030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:13.545437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf329-dc89-41eb-a36c-d6136e9eb492 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:13.561669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:13.562326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.562609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.562879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.579141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:13.582139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:13.601280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 814/1646] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Aug 1 14:20:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:13.609193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1e9b0-b668-439b-96ce-869c712deec7 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.609599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1e9b0-b668-439b-96ce-869c712deec7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.609948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1e9b0-b668-439b-96ce-869c712deec7 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.615155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 815/1647] 10.222.0.17 () {54 vars in 912 bytes} [Mon Aug 1 14:20:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:13.620361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.620646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.620748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 833/1648] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:13.621036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.660149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:13.662117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:13.695173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 816/1649] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:20:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:13.749602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:13.756983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:13.766414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848939-07d4-4987-8ace-5109ee5f2769 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:13.772514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:13.774130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001d0f5-1038-4b79-8200-f9877cf2ff17 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:13.774574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.775038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.775550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.784186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:13.785116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.786450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.787136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:13.821738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:13.825698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:13.868940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:13.872115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:13.875577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 817/1650] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:13.913028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 834/1651] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:13.959158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:13.979401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d427c-da1a-46ba-ac7d-48ce6a1b2ae9 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:13.994907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f543f40-3d3c-4d14-a947-ef3b11bf7d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:13.994907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f543f40-3d3c-4d14-a947-ef3b11bf7d30 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:13.994907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f543f40-3d3c-4d14-a947-ef3b11bf7d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:13.997583 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f543f40-3d3c-4d14-a947-ef3b11bf7d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:14.174761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f543f40-3d3c-4d14-a947-ef3b11bf7d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:14.176203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f543f40-3d3c-4d14-a947-ef3b11bf7d30 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:14.203614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 818/1652] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:15.353821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:15.588817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56f76f-319c-4d8a-a0cc-0b22fdfc2cce service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:15.608385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:15.609068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.609356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.609654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] PATH_INFO: `/v3/users/ea7fd8ddcade4a279c59c8545eaa76d5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.610946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7fd8ddcade4a279c59c8545eaa76d5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:15.612973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:15.669943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:15.724713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7fd8ddcade4a279c59c8545eaa76d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:15.728753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7fd8ddcade4a279c59c8545eaa76d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:15.755130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7fd8ddcade4a279c59c8545eaa76d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:15.774375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7fd8ddcade4a279c59c8545eaa76d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:15.783594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7fd8ddcade4a279c59c8545eaa76d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:15.800114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 835/1653] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:15 2022] DELETE /identity/v3/users/ea7fd8ddcade4a279c59c8545eaa76d5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:15.805999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:15.806635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:15.806635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:15.851683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:15.852951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:15.990782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 819/1654] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:16.038072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:16.040420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:16.162864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d995a-3ee1-4a54-aa5d-989b9aba9389 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:16.168035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:16.169675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:16.171031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:16.172033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:16.202249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:16.204067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:16.219900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 836/1655] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:18.720189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:18.721507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:18.747718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2536d-ae67-403a-a8cb-faa86dfcadfd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:18.775191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:18.775900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:18.776185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:18.776465 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] PATH_INFO: `/v3/projects/26548c4abfb7420c97b621a1a8bfa196` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:18.779039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] RBAC: Authorizing `identity:delete_project(project_id=26548c4abfb7420c97b621a1a8bfa196)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:18.789378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:18.845061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:18.931896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26548c4abfb7420c97b621a1a8bfa196', 'request_id': 'req-d3129c69-ac76-44b3-94b7-8015b600355c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:18.938066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26548c4abfb7420c97b621a1a8bfa196', 'request_id': 'req-d3129c69-ac76-44b3-94b7-8015b600355c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:18.966545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26548c4abfb7420c97b621a1a8bfa196', 'request_id': 'req-d3129c69-ac76-44b3-94b7-8015b600355c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:18.998840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] The token cache is being invalidate because project 26548c4abfb7420c97b621a1a8bfa196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:18.999225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.004744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3129c69-ac76-44b3-94b7-8015b600355c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.009486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 820/1656] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:18 2022] DELETE /identity/v3/projects/26548c4abfb7420c97b621a1a8bfa196 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:19.009895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:19.020766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611818bb-74b1-4f39-8622-412ddb21d83b service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:19.171385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:19.172322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:19.172560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:19.172997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] PATH_INFO: `/v3/users/7afa9896b553423b9d424a389b5f877e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:19.174944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7afa9896b553423b9d424a389b5f877e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:19.176933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:19.301482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:19.305209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7afa9896b553423b9d424a389b5f877e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.306308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7afa9896b553423b9d424a389b5f877e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.338043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7afa9896b553423b9d424a389b5f877e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.357033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7afa9896b553423b9d424a389b5f877e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.376433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7afa9896b553423b9d424a389b5f877e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:19.385905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 837/1657] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:19 2022] DELETE /identity/v3/users/7afa9896b553423b9d424a389b5f877e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:20.363144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:20.363765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:20.363765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:20.490748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] MFA Rules not processed for user `9eda4e19122444cf9cb2b6941adff9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:20.492563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:20.651951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:20.653491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:20.786259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 821/1658] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:20.803707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:20.805628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:20.862869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82778cc-ce28-4599-9fc6-f1ab77e2cce0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:20.878056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:20.881921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:20.882509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:20.882928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] PATH_INFO: `/v3/projects/289056bc50f9476ca8d1445fbd61a622` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:20.889152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=289056bc50f9476ca8d1445fbd61a622)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:20.890287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:20:20.890287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 01 14:20:20.890287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: Aug 01 14:20:20.893299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:21.052758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289056bc50f9476ca8d1445fbd61a622', 'request_id': 'req-d1092a8f-bb11-4998-a26a-006ac51884d4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:21.071515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289056bc50f9476ca8d1445fbd61a622', 'request_id': 'req-d1092a8f-bb11-4998-a26a-006ac51884d4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:21.085684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c47c3-eb88-40fd-aec9-86e4786ba6e2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:21.094783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289056bc50f9476ca8d1445fbd61a622', 'request_id': 'req-d1092a8f-bb11-4998-a26a-006ac51884d4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:21.129128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] The token cache is being invalidate because project 289056bc50f9476ca8d1445fbd61a622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:21.129559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:21.131023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d1092a8f-bb11-4998-a26a-006ac51884d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:21.136912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 838/1659] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:20 2022] DELETE /identity/v3/projects/289056bc50f9476ca8d1445fbd61a622 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:21.138716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06825a3-c484-4844-80d3-eaecda1cc302 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.139088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06825a3-c484-4844-80d3-eaecda1cc302 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.139544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06825a3-c484-4844-80d3-eaecda1cc302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.185308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:21.189432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f06825a3-c484-4844-80d3-eaecda1cc302 None None] MFA Rules not processed for user `67fcd16d046c4df2a5dc99b43262375a`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:21.198452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f06825a3-c484-4844-80d3-eaecda1cc302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:21.352104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:21.353924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:21.357548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.357728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.358012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.380040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:21.453725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 839/1660] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:21.455360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91725888-90a4-40b8-a393-69594de70a77 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.455694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91725888-90a4-40b8-a393-69594de70a77 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.455974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91725888-90a4-40b8-a393-69594de70a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.517516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-91725888-90a4-40b8-a393-69594de70a77 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:21.518092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-91725888-90a4-40b8-a393-69594de70a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:21.589855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:21.591796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b84be-cbc0-4265-aec8-54b1e0eb9fdc service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:21.632038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 822/1661] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 829 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:21.637748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5b4e8-7ccf-4c1a-89be-61468f828001 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.638057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5b4e8-7ccf-4c1a-89be-61468f828001 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.638348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5b4e8-7ccf-4c1a-89be-61468f828001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.704013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2ed5b4e8-7ccf-4c1a-89be-61468f828001 None None] MFA Rules not processed for user `67fcd16d046c4df2a5dc99b43262375a`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:21.705837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ed5b4e8-7ccf-4c1a-89be-61468f828001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:21.725036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 823/1662] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:21.726696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 840/1663] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:21.737844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91725888-90a4-40b8-a393-69594de70a77 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:21.739075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.739305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-91725888-90a4-40b8-a393-69594de70a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:21.739424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.740241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.778254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91725888-90a4-40b8-a393-69594de70a77 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:21.795533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:21.796917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:21.797775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:21.799271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.799814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.800250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.820244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 824/1664] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:21.821693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:21.822891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:21.824964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:21.843254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31652cb-3e6c-474a-bcea-a00ad63d4dca None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:21.853416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 841/1665] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:21.881416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:21.881416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.881416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.881416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.882078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:21.882762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:21.887990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:21.932254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 825/1666] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:21.938898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:21.956842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14fe0-0908-42ae-9d1e-ba26c549835f service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:21.984882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:21.986084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:21.986431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:21.986898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:21.987701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:21.990284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:22.112390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:22.120546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 842/1667] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:21 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:22.126827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214ab22-1ba2-4a6c-8515-2ef595f2b5ec admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:22.355274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:22.355950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:22.356508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:22.357176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:22.358886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:22.365827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:22.408779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 826/1668] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:22 2022] POST /identity/v3/users => generated 287 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:22.413474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:22.435044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6abea-378e-43f0-b270-8db64168f901 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:22.456542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:22.457265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:22.457621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:22.457859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:22.459077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:22.462141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:22.499156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:22.500187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 843/1669] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:22.514941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c12a6-4909-4ea5-a321-321c70e4e67e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:22.541784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e4f7a-2049-4788-b7d5-2db2794b1a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:22.542425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e4f7a-2049-4788-b7d5-2db2794b1a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:22.542711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e4f7a-2049-4788-b7d5-2db2794b1a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:22.542988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e4f7a-2049-4788-b7d5-2db2794b1a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:22.543771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e4f7a-2049-4788-b7d5-2db2794b1a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:22.545403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e4f7a-2049-4788-b7d5-2db2794b1a85 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:22.584857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 827/1670] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:22.586461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:22.603791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b9d82-3640-469e-81a5-e9b0be38948d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:22.619405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:22.620144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:22.620429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:22.620878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] PATH_INFO: `/v3/projects/f79f01ade73b4ca0856b58cb55e51a33/users/f47dab911b9545be95d7e985001eaed9/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:22.659600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79f01ade73b4ca0856b58cb55e51a33, user_id=f47dab911b9545be95d7e985001eaed9, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:22.663008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:22.690705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 844/1671] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:22 2022] PUT /identity/v3/projects/f79f01ade73b4ca0856b58cb55e51a33/users/f47dab911b9545be95d7e985001eaed9/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:22.702926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a485c-4cc9-4291-a561-c98057e297a6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:22.704883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a485c-4cc9-4291-a561-c98057e297a6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:22.705759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a485c-4cc9-4291-a561-c98057e297a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:22.762729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-cd4a485c-4cc9-4291-a561-c98057e297a6 None None] MFA Rules not processed for user `f47dab911b9545be95d7e985001eaed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:22.764807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cd4a485c-4cc9-4291-a561-c98057e297a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:22.783389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:22.886463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 828/1672] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:22.891222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:22.980565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b3afc-4226-4a75-a7fb-b218cac16f20 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:22.981049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b3afc-4226-4a75-a7fb-b218cac16f20 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:22.981605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b3afc-4226-4a75-a7fb-b218cac16f20 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:22.983077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 829/1673] 10.222.0.17 () {54 vars in 912 bytes} [Mon Aug 1 14:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:23.007203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.008237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.008838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.092580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8af95-997b-416a-bd3e-367944fd4aa6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:23.113023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:23.113872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.117756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.118214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.119083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:23.121530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:23.129104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] MFA Rules not processed for user `771f49a069014fdc956e073c2d0a0fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:23.135224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:23.166439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:23.204133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 845/1674] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:22 2022] POST /identity/v3/projects => generated 373 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:23.208311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:23.368137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 830/1675] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:20:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:23.383365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:23.384867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:23.402435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103faf3-5173-487e-b584-1343363aeb21 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:23.410644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8795fb-44e2-4d4e-86b7-6948c2137d14 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:23.425656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:23.426415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.426781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.427703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.430621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:23.432936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:23.526047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 846/1676] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:23 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:23.539096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:23.555799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:23.556978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.557412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.557827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.576348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:23.577959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:23.584429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6652c6-dc5c-4bb6-946d-c70ca259b118 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:23.607867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 831/1677] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:23.618803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:23.619558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.622846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.623171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.629350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:23.636495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:23.683066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:23.685341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 847/1678] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:23.721410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bb141-7c8a-425c-8350-8f5579583002 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:23.738369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f588069-7f65-4dbe-86d1-c910af88e047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:23.739328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588069-7f65-4dbe-86d1-c910af88e047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.739689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588069-7f65-4dbe-86d1-c910af88e047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.739975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588069-7f65-4dbe-86d1-c910af88e047 admin admin] PATH_INFO: `/v3/projects/d60360e0c25d4336aec33a4ffac83558/users/8b96a1739f8045bd8bb36e80a616a8bb/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.775091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f588069-7f65-4dbe-86d1-c910af88e047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60360e0c25d4336aec33a4ffac83558, user_id=8b96a1739f8045bd8bb36e80a616a8bb, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:23.778195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f588069-7f65-4dbe-86d1-c910af88e047 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:23.803322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 832/1679] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:23 2022] PUT /identity/v3/projects/d60360e0c25d4336aec33a4ffac83558/users/8b96a1739f8045bd8bb36e80a616a8bb/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:23.807759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:23.826240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0980b4b-bc3d-4863-8fe6-0139c384e36a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:23.902700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d2a3-b503-4302-9008-448a863a671e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.903147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d2a3-b503-4302-9008-448a863a671e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.903246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d2a3-b503-4302-9008-448a863a671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.964010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6b76d2a3-b503-4302-9008-448a863a671e None None] MFA Rules not processed for user `fbaa67d901e34419b1616a8457fbd683`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:23.965281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6b76d2a3-b503-4302-9008-448a863a671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:23.966709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:23.967297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:23.968341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:23.968341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:23.969023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:23.970680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:24.003927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:24.015830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 848/1680] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:24.018609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:24.041276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb43b52-5805-4e50-ad0e-6547307820c0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:24.079484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:24.081869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:24.083035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:24.083035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] PATH_INFO: `/v3/projects/d60360e0c25d4336aec33a4ffac83558/users/8b96a1739f8045bd8bb36e80a616a8bb/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:24.092659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60360e0c25d4336aec33a4ffac83558, user_id=8b96a1739f8045bd8bb36e80a616a8bb, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:24.109046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:24.125123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 833/1681] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:24.136909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 849/1682] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:24 2022] PUT /identity/v3/projects/d60360e0c25d4336aec33a4ffac83558/users/8b96a1739f8045bd8bb36e80a616a8bb/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:24.148148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:24.148453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:24.148828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:24.158959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:24.160787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:24.216625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] MFA Rules not processed for user `8b96a1739f8045bd8bb36e80a616a8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:24.218102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:24.320609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:24.379802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e528-df76-44ca-acc2-9656a55f8432 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:24.411958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10016b5a-a589-4fa5-8d6b-325697cb9af3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:24.413009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10016b5a-a589-4fa5-8d6b-325697cb9af3 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:24.413488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10016b5a-a589-4fa5-8d6b-325697cb9af3 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:24.413979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10016b5a-a589-4fa5-8d6b-325697cb9af3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:24.420920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 834/1683] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:24.460111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:24.462699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:24.580993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d09bc-0d33-47ce-9e1a-616fafbaa5e2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:24.586520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc8cc8-2346-464d-9995-f026b8882e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:24.587649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8cc8-2346-464d-9995-f026b8882e1c service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:24.588197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8cc8-2346-464d-9995-f026b8882e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:24.589005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8cc8-2346-464d-9995-f026b8882e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:24.613273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc8cc8-2346-464d-9995-f026b8882e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:24.618915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc8cc8-2346-464d-9995-f026b8882e1c service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:24.636940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10016b5a-a589-4fa5-8d6b-325697cb9af3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:24.638571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10016b5a-a589-4fa5-8d6b-325697cb9af3 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:24.641195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 835/1684] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:24.664121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 850/1685] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:24.712094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ab598-c775-444e-ab19-b2f63a6a41f3 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:24.712488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ab598-c775-444e-ab19-b2f63a6a41f3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:24.713572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ab598-c775-444e-ab19-b2f63a6a41f3 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:24.715316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 836/1686] 10.222.0.17 () {54 vars in 961 bytes} [Mon Aug 1 14:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:24.724226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:24.724790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:24.725204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:24.801146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:24.803344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:25.054694 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 851/1687] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Aug 1 14:20:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:25.064350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ab598-c775-444e-ab19-b2f63a6a41f3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:25.104087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ab598-c775-444e-ab19-b2f63a6a41f3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:25.125483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:25.126091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:25.126576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:25.127014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:25.142554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:25.163722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:25.167151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83beff-55f7-47dc-bce6-f92b544ac106 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:25.181921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 837/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:25.869999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:25.871556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:26.056083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f88e-e4d7-43bf-9517-5ef0de763b36 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:26.086454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:26.088109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:26.088647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:26.089431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] PATH_INFO: `/v3/users/e47a94fa5e554608aadafdabadd901b9` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:26.092766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47a94fa5e554608aadafdabadd901b9)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:26.100612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:26.189884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:26.249133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47a94fa5e554608aadafdabadd901b9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:26.249867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47a94fa5e554608aadafdabadd901b9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:26.293084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47a94fa5e554608aadafdabadd901b9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:26.321679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47a94fa5e554608aadafdabadd901b9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:26.356714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-530548f8-0829-4ef3-a1a3-1917c4f326c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47a94fa5e554608aadafdabadd901b9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:26.381724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 852/1689] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:25 2022] DELETE /identity/v3/users/e47a94fa5e554608aadafdabadd901b9 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:26.385564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:26.386650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:26.387035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:26.446354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:26.454198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:26.560973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:26.637493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:26.637655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:26.637901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:26.691612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 838/1690] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:26.701812 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] MFA Rules not processed for user `9eda4e19122444cf9cb2b6941adff9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:26.707105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:26.708911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:26.711519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:26.813984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 853/1691] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:26.820558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb3eca-3849-4e7a-983c-c7a52951ec81 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:26.828652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48773273-d642-44ad-ae11-079bbdb626ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:26.830098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48773273-d642-44ad-ae11-079bbdb626ba service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:26.830502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48773273-d642-44ad-ae11-079bbdb626ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:26.830910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48773273-d642-44ad-ae11-079bbdb626ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:26.842878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48773273-d642-44ad-ae11-079bbdb626ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:26.844581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48773273-d642-44ad-ae11-079bbdb626ba service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:26.866876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 839/1692] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:27.078402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:27.080038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:27.108950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb66a94-d544-4f11-9515-be6c3b0e35c2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:27.137477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:27.140579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:27.140664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:27.145615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] PATH_INFO: `/v3/users/f47dab911b9545be95d7e985001eaed9` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:27.154537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47dab911b9545be95d7e985001eaed9)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:27.156901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:27.322463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47dab911b9545be95d7e985001eaed9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:27.323597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47dab911b9545be95d7e985001eaed9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:27.367009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47dab911b9545be95d7e985001eaed9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:27.392055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47dab911b9545be95d7e985001eaed9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:27.418079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47dab911b9545be95d7e985001eaed9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:27.430056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 854/1693] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:27 2022] DELETE /identity/v3/users/f47dab911b9545be95d7e985001eaed9 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:27.434840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:27.436018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:27.437266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:27.483777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:27.486070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:27.631783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 840/1694] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:27.675131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:27.677061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:27.774544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:27.841923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379df252-b0f8-45ae-a046-4e518aef73c1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:27.850737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d3c9f-a1e6-432d-8189-f2d06859de66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:27.851435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d3c9f-a1e6-432d-8189-f2d06859de66 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:27.851755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d3c9f-a1e6-432d-8189-f2d06859de66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:27.852033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d3c9f-a1e6-432d-8189-f2d06859de66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:27.880902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d3c9f-a1e6-432d-8189-f2d06859de66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:27.890675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d3c9f-a1e6-432d-8189-f2d06859de66 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:27.926473 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 855/1695] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:28.466917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:28.468502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:28.496275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4ebb2-8996-4a5b-8bc9-94bddbb45f4f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:28.510755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:28.513172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:28.513642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:28.514062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] PATH_INFO: `/v3/projects/a07d5eb233d349eab325368d9914f3e9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:28.517711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07d5eb233d349eab325368d9914f3e9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:28.524701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:28.612208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a981ca-a177-417c-a19c-d5f90e634037 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:28.612818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a981ca-a177-417c-a19c-d5f90e634037 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:28.612818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a981ca-a177-417c-a19c-d5f90e634037 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:28.615169 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 856/1696] 10.222.0.17 () {54 vars in 912 bytes} [Mon Aug 1 14:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:28.626963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:28.627258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:28.627567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:28.702495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07d5eb233d349eab325368d9914f3e9', 'request_id': 'req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:28.731770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] MFA Rules not processed for user `771f49a069014fdc956e073c2d0a0fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:28.734036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:28.747886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07d5eb233d349eab325368d9914f3e9', 'request_id': 'req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:28.774490 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07d5eb233d349eab325368d9914f3e9', 'request_id': 'req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:28.779825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:28.830538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] The token cache is being invalidate because project a07d5eb233d349eab325368d9914f3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:28.830840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:28.838649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-84f0dac8-b5d8-4f73-83a6-4c619c316cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:28.852243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 841/1697] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:28 2022] DELETE /identity/v3/projects/a07d5eb233d349eab325368d9914f3e9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:28.859007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12b9c8-d193-47f5-8e1c-3228e1b29e3f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:28.860598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12b9c8-d193-47f5-8e1c-3228e1b29e3f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:28.860894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12b9c8-d193-47f5-8e1c-3228e1b29e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:28.941892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-be12b9c8-d193-47f5-8e1c-3228e1b29e3f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:28.943271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-be12b9c8-d193-47f5-8e1c-3228e1b29e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:28.964991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 857/1698] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:20:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:29.017528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.019065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:29.034792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4fce-152d-419d-b226-c0dc907ff2d5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:29.043912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:29.047706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 842/1699] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:29.053893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.055231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.055985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.057651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.058154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.065311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.090219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:29.098594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:29.129036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:29.130356 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:29.135288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 858/1700] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:29.179977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 843/1701] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:29.183017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.232610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adee9b0-38bd-4db5-8d8b-dc70ab179ec6 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:29.246237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:29.247961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.248756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.249402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.251054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:29.252003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:29.256302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:29.292536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 859/1702] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:29.300541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.332342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc2e7c-887c-4dcf-ac16-79581f5832fc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:29.346172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d83bc-5fc2-4080-b29f-6a2e57a1590b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:29.348044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d83bc-5fc2-4080-b29f-6a2e57a1590b admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.348726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d83bc-5fc2-4080-b29f-6a2e57a1590b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.349297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d83bc-5fc2-4080-b29f-6a2e57a1590b admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.351076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d83bc-5fc2-4080-b29f-6a2e57a1590b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:29.356275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d83bc-5fc2-4080-b29f-6a2e57a1590b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:29.484509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 844/1703] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:29 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:29.489026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.533945 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.538273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.538702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.543705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128addf-173a-4bf4-ae1c-b04211e0f7d2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:29.748803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:29.757031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:29.784454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:29.785614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.785614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.789678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.792565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:29.799146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:29.867432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 860/1704] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:29 2022] POST /identity/v3/users => generated 287 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:29.874140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.904781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4f917-1e6a-4074-8821-ba478854d20e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:29.917246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:29.918250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:29.919885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:29.921149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:29.922657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:29.927404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:29.961379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 861/1705] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:29.961630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 845/1706] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:29.962698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:29.983198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371fe1e-f626-481e-afe3-f894c164f4af admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:30.008497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269825-0481-4279-a75e-7c35dfae321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:30.010175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269825-0481-4279-a75e-7c35dfae321d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.010175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269825-0481-4279-a75e-7c35dfae321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.023815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269825-0481-4279-a75e-7c35dfae321d admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.027062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54269825-0481-4279-a75e-7c35dfae321d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:30.034186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54269825-0481-4279-a75e-7c35dfae321d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:30.066111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 862/1707] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:30.072175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:30.090031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c62b-f371-4fe7-95d4-a78cf10a1a2c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.091333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c62b-f371-4fe7-95d4-a78cf10a1a2c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.092062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c62b-f371-4fe7-95d4-a78cf10a1a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.111862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6981be-9de5-48c5-931f-d43ad9a2d678 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:30.136191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:30.136973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.139625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.139975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] PATH_INFO: `/v3/projects/ee3d21cda75742ef9ee0aceb194da05e/users/d14d32969d0d45f59a9f5f2b8c367e13/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.178455 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6a80c62b-f371-4fe7-95d4-a78cf10a1a2c None None] MFA Rules not processed for user `fbaa67d901e34419b1616a8457fbd683`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:30.180088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6a80c62b-f371-4fe7-95d4-a78cf10a1a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.198115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3d21cda75742ef9ee0aceb194da05e, user_id=d14d32969d0d45f59a9f5f2b8c367e13, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:30.208136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:30.232287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:30.244532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 846/1708] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:30 2022] PUT /identity/v3/projects/ee3d21cda75742ef9ee0aceb194da05e/users/d14d32969d0d45f59a9f5f2b8c367e13/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:30.247369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:30.250526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.323493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:30.351215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 863/1709] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:30.351215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.351215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.351215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.414127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] MFA Rules not processed for user `d14d32969d0d45f59a9f5f2b8c367e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:30.415610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.480378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4764ea7-d1e8-48c8-a2e6-28a58f8b83b7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:30.506266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:30.508814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.513593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.522382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] PATH_INFO: `/v3/projects/f79f01ade73b4ca0856b58cb55e51a33` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.526561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79f01ade73b4ca0856b58cb55e51a33)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:30.537928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:30.647408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79f01ade73b4ca0856b58cb55e51a33', 'request_id': 'req-a62f7af4-a674-4270-b36d-78297935b44c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:30.657511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 864/1710] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:30.663811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:30.665176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79f01ade73b4ca0856b58cb55e51a33', 'request_id': 'req-a62f7af4-a674-4270-b36d-78297935b44c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:30.667582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.692294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79f01ade73b4ca0856b58cb55e51a33', 'request_id': 'req-a62f7af4-a674-4270-b36d-78297935b44c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:30.725414 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] The token cache is being invalidate because project f79f01ade73b4ca0856b58cb55e51a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:30.726051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:30.727834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:30.730870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 847/1711] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:30 2022] DELETE /identity/v3/projects/f79f01ade73b4ca0856b58cb55e51a33 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:30.735394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:30.737755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:30.957238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7af4-a674-4270-b36d-78297935b44c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:30.965056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32323e-7995-45f8-a68e-3884376e021d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:30.971779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:30.972829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:30.973181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:30.973764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] PATH_INFO: `/v3/users/8b96a1739f8045bd8bb36e80a616a8bb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:30.975292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b96a1739f8045bd8bb36e80a616a8bb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:30.982495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:20:30.982495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 01 14:20:30.982495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: Aug 01 14:20:30.988761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:31.034409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:31.104091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:31.105483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:31.106208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:31.106801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:31.110321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:31.212810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b96a1739f8045bd8bb36e80a616a8bb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:31.214149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b96a1739f8045bd8bb36e80a616a8bb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:31.216439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:31.292203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b96a1739f8045bd8bb36e80a616a8bb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:31.348296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b96a1739f8045bd8bb36e80a616a8bb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:31.349592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b96a1739f8045bd8bb36e80a616a8bb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:31.379364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 848/1712] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:30 2022] DELETE /identity/v3/users/8b96a1739f8045bd8bb36e80a616a8bb => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:31.380914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:31.382602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:31.384643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:31.386400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:31.401653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 865/1713] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:20:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:20:31.537812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4547a9-1cd0-4abb-93b5-529730a4c0d2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:31.556468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:31.557349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:31.557647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:31.559047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:31.566674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:31.740336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:31.744004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:31.773796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae469d-37fe-4897-a852-16689b61450b service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:31.832652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:31.843325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f0312-c413-4e05-a36d-a4ebabfad7f8 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:31.899568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 849/1714] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:31.949910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:31.950656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:31.951829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:31.952123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] PATH_INFO: `/v3/users/f872c3471866499a9d9e2d2561f4a4e0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:31.955802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f872c3471866499a9d9e2d2561f4a4e0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:31.978212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:32.171604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:32.171926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:32.172275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:32.233093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f872c3471866499a9d9e2d2561f4a4e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:32.237458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f872c3471866499a9d9e2d2561f4a4e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:32.252826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] MFA Rules not processed for user `771f49a069014fdc956e073c2d0a0fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:32.254323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:32.323693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f872c3471866499a9d9e2d2561f4a4e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:32.350202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f872c3471866499a9d9e2d2561f4a4e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:32.391988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f148d2e-2814-4bf6-8568-b708b4056ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f872c3471866499a9d9e2d2561f4a4e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:32.399040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:32.427498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 866/1715] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:31 2022] DELETE /identity/v3/users/f872c3471866499a9d9e2d2561f4a4e0 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:32.450885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:32.455117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:32.458106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:32.588157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 850/1716] 10.222.0.69 () {58 vars in 1015 bytes} [Mon Aug 1 14:20:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:32.599736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:32.605955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:32.719850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:32.733500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:32.764491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc7ad6-18d1-4002-b4c3-4a0ed7baedf2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:32.782061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:32.885163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:32.886150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:32.886610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:32.887086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:32.912449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:32.914828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:32.934759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 851/1717] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:33.021241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 867/1718] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 590 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:33.170462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:33.172351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:33.202807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644c522-0fe0-49e1-b3fa-8190815385b2 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:33.225259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ee083-e92e-4bb3-87fd-7142edd7c182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:33.226180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ee083-e92e-4bb3-87fd-7142edd7c182 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:33.230213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ee083-e92e-4bb3-87fd-7142edd7c182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:33.231204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ee083-e92e-4bb3-87fd-7142edd7c182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:33.282950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ee083-e92e-4bb3-87fd-7142edd7c182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:33.295250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ee083-e92e-4bb3-87fd-7142edd7c182 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:33.350496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 852/1719] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:33.702571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:33.706432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:33.756767 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8025f3-1493-4954-9c19-2fbb44e4ceaa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:33.768989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:33.771759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:33.772765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:33.773225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] PATH_INFO: `/v3/projects/d60360e0c25d4336aec33a4ffac83558` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:33.776601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60360e0c25d4336aec33a4ffac83558)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:33.779878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:33.966664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60360e0c25d4336aec33a4ffac83558', 'request_id': 'req-fe104f23-57de-4864-8d09-2a43b0165c83'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:34.001843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60360e0c25d4336aec33a4ffac83558', 'request_id': 'req-fe104f23-57de-4864-8d09-2a43b0165c83'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:34.036410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60360e0c25d4336aec33a4ffac83558', 'request_id': 'req-fe104f23-57de-4864-8d09-2a43b0165c83'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:34.088787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] The token cache is being invalidate because project d60360e0c25d4336aec33a4ffac83558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:34.091935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:34.095114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe104f23-57de-4864-8d09-2a43b0165c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:34.105669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 868/1720] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:33 2022] DELETE /identity/v3/projects/d60360e0c25d4336aec33a4ffac83558 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:34.109732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:34.110215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:34.110505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:34.151239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:34.153681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:34.291450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 853/1721] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:34.292624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:34.293035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:34.293649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:34.376826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:34.380477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:34.428803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 869/1722] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:34.432980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:34.463058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775eab0-2462-453e-9d50-2288ebe9e4bb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:34.483135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:34.484244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:34.486036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:34.488630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:34.494500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:34.496933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:34.505381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:34.556507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:34.564138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 854/1723] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:34.570179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48226228-5559-43e4-9f00-b51d466b24fa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:34.588112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:34.588925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:34.590967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:34.593530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:34.595204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:34.599161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:34.702196 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 870/1724] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:34.706019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:34.718314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c88f8c-6c13-49c3-b121-0dafe81ed392 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:35.012547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:35.016301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:35.048307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:35.050470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.051882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.052431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.053768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:35.057785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:35.127233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 855/1725] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:34 2022] POST /identity/v3/users => generated 294 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:35.129372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:35.173275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594f6ae-26ba-4be5-84bb-98c30b78e5aa admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:35.191326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:35.192633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.196346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.197915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.200573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:35.203555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:35.210119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:35.242554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266f9c-d36b-49cc-aede-ccd8195268e2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:35.252707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 856/1726] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:35.254807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:35.262087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:35.262771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.263194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.263313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] PATH_INFO: `/v3/projects/638b409b76a94100a7efeb925b86ea6f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.272534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] RBAC: Authorizing `identity:delete_project(project_id=638b409b76a94100a7efeb925b86ea6f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:35.275230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:35.284212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934b266-685c-461b-afd8-b23fe0f85dde admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:35.301109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:35.302293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.303785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.305617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.308232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:35.310813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:35.314502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:35.335795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 857/1727] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:35.337067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:35.384383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cbcf3-61a3-4488-acde-b446bb074d7a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:35.403622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e2b33-26ea-4871-adef-f52475217462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:35.407960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e2b33-26ea-4871-adef-f52475217462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.417393 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e2b33-26ea-4871-adef-f52475217462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.418128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e2b33-26ea-4871-adef-f52475217462 admin admin] PATH_INFO: `/v3/projects/c73be4b269834dc68b46600f1e34d957/users/3eb9c8687dfc4847aa4717d0df66f20c/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.482508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638b409b76a94100a7efeb925b86ea6f', 'request_id': 'req-930adfab-0004-4b6f-8549-420d054ce50f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:35.490593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e2b33-26ea-4871-adef-f52475217462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c73be4b269834dc68b46600f1e34d957, user_id=3eb9c8687dfc4847aa4717d0df66f20c, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:35.502300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e2b33-26ea-4871-adef-f52475217462 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:35.511570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638b409b76a94100a7efeb925b86ea6f', 'request_id': 'req-930adfab-0004-4b6f-8549-420d054ce50f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:35.538871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638b409b76a94100a7efeb925b86ea6f', 'request_id': 'req-930adfab-0004-4b6f-8549-420d054ce50f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:35.546797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 858/1728] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:35 2022] PUT /identity/v3/projects/c73be4b269834dc68b46600f1e34d957/users/3eb9c8687dfc4847aa4717d0df66f20c/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:35.555120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6be97-e9bf-45e9-835a-c0a0f005e601 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.555635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6be97-e9bf-45e9-835a-c0a0f005e601 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.556033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6be97-e9bf-45e9-835a-c0a0f005e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.576484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] The token cache is being invalidate because project 638b409b76a94100a7efeb925b86ea6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:35.578990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:35.583437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-930adfab-0004-4b6f-8549-420d054ce50f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:35.586454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 871/1729] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:35 2022] DELETE /identity/v3/projects/638b409b76a94100a7efeb925b86ea6f => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:35.592564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.599440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.601174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.662813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e0c6be97-e9bf-45e9-835a-c0a0f005e601 None None] MFA Rules not processed for user `3eb9c8687dfc4847aa4717d0df66f20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:35.664951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e0c6be97-e9bf-45e9-835a-c0a0f005e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:35.670435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:35.671649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:35.946938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 872/1730] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:35.947118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 859/1731] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:35.948399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:35.954056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:35.955077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:35.955513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:35.955889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:35.987431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d322e-0fe8-4a9e-8065-673b1a77fc4a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.007694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:36.007694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:36.009463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:36.010215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.010609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.011012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:36.011959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:36.014558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:36.038862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 860/1732] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:36.039555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:36.060168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1acef-b0fb-4466-bfe6-27e8ea457059 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.068371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 873/1733] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:35 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:36.072263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:36.075860 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:36.076467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.076746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.077021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:36.077723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:36.078425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:36.082750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:36.112387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 861/1734] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:36.116570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:36.138939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8258ae-ae57-4b36-adc6-6fd9288d7b9a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.272993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:36.273929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.274387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.274809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:36.276377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:36.282716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:36.402988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 862/1735] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:36 2022] POST /identity/v3/projects => generated 387 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:36.404389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:36.436323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37807ac6-ffe1-4322-aecf-0b433b181ac1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.464468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:36.566338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:36.714359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0cf6a-6638-4dec-855b-8d80d9251fd9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.742534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9625-115f-4411-9ba6-b6ff97b64bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:36.743053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9625-115f-4411-9ba6-b6ff97b64bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.743334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9625-115f-4411-9ba6-b6ff97b64bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.743649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9625-115f-4411-9ba6-b6ff97b64bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:36.744344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d9625-115f-4411-9ba6-b6ff97b64bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:36.746333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d9625-115f-4411-9ba6-b6ff97b64bfa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:36.775186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:36.776007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.776358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.776704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:36.777714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:36.780104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:36.858374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 863/1736] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:36 2022] POST /identity/v3/users => generated 294 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:36.863923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98935a-adae-42d9-beff-372feda73938 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.872278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98935a-adae-42d9-beff-372feda73938 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.873673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98935a-adae-42d9-beff-372feda73938 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:36.877348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 864/1737] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:20:36 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:36.880015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98935a-adae-42d9-beff-372feda73938 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:36.887197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0c98935a-adae-42d9-beff-372feda73938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:36.895036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 874/1738] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:36 2022] POST /identity/v3/users => generated 299 bytes in 823 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:36.897498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:36.972350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98935a-adae-42d9-beff-372feda73938 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.986932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680874e-9c33-4c4e-88cd-0ef6a6036256 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:36.990774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:36.993709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:36.994249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:36.995342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] PATH_INFO: `/v3/users/d14d32969d0d45f59a9f5f2b8c367e13` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.002238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14d32969d0d45f59a9f5f2b8c367e13)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.009893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.011685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:37.012446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.012870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.013213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.014003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.015656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.066886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 875/1739] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:37.067821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.143995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96906909-89b8-49e1-bce7-0759deccca2d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:37.160305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba172f-bc01-4379-ac19-634db26cdcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:37.162196 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba172f-bc01-4379-ac19-634db26cdcb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.162936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba172f-bc01-4379-ac19-634db26cdcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.164481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba172f-bc01-4379-ac19-634db26cdcb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.166085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba172f-bc01-4379-ac19-634db26cdcb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.169058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba172f-bc01-4379-ac19-634db26cdcb9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.207046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14d32969d0d45f59a9f5f2b8c367e13'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:37.210467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14d32969d0d45f59a9f5f2b8c367e13'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:37.210971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 876/1740] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:37.214537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.222414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.224078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.270348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14d32969d0d45f59a9f5f2b8c367e13'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:37.287438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:37.289572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:37.292915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14d32969d0d45f59a9f5f2b8c367e13'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:37.294181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14d32969d0d45f59a9f5f2b8c367e13'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:37.337593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 865/1741] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:36 2022] DELETE /identity/v3/users/d14d32969d0d45f59a9f5f2b8c367e13 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:37.340747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.351165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:37.456378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 877/1742] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:37.457255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.458631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:37.473632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712c370-49a5-4e9c-b38f-3008a6058c75 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:37.539283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833168c-4eca-4df8-b5e8-1f633eeba0eb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:37.553683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac15e6f-2963-4ee3-9f51-ebb5894fff94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:37.557332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac15e6f-2963-4ee3-9f51-ebb5894fff94 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.557332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac15e6f-2963-4ee3-9f51-ebb5894fff94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.557332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac15e6f-2963-4ee3-9f51-ebb5894fff94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.561298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac15e6f-2963-4ee3-9f51-ebb5894fff94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.566023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac15e6f-2963-4ee3-9f51-ebb5894fff94 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.602240 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 866/1743] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:37.607329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.607639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.608698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.626045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:37.628852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.629165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.629443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] PATH_INFO: `/v3/projects/ca816f5b04454fc49899eb67b8d4f4a3/users/57c5f279d69140749dae4c1f3dcec10e/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.675997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:37.681687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:37.686741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:37.688049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca816f5b04454fc49899eb67b8d4f4a3, user_id=57c5f279d69140749dae4c1f3dcec10e, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.691024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.701677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 867/1744] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:37.705081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.717268 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 878/1745] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:37 2022] PUT /identity/v3/projects/ca816f5b04454fc49899eb67b8d4f4a3/users/57c5f279d69140749dae4c1f3dcec10e/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:37.719942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.735661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63849-a448-43f9-81fc-d98af89d9bdf None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:37.766482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:37.871200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:37.872139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.876080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.876621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] PATH_INFO: `/v3/projects/bffb60f1c47947158b338e6eb5aa731c/users/f9e85a31297b4b2aad6eb597ba47d6a5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.879288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ca322-a0bc-4484-bd0e-efd24caa7010 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:37.885744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:37.886620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:37.887055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:37.887483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:37.888345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.891535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.906498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 879/1746] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:37.906865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffb60f1c47947158b338e6eb5aa731c, user_id=f9e85a31297b4b2aad6eb597ba47d6a5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:37.907757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.910228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:37.911544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:37.938464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 868/1747] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:37 2022] PUT /identity/v3/projects/bffb60f1c47947158b338e6eb5aa731c/users/f9e85a31297b4b2aad6eb597ba47d6a5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:37.940115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:37.944395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:38.015193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8549636-d610-48e1-bce0-c829a62fe70e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:38.087776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c2947-1605-4132-95fa-3603fd9a648a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:38.094585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5636e-13e7-4350-9e91-a4e143268324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:38.096154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5636e-13e7-4350-9e91-a4e143268324 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:38.096791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:38.097420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5636e-13e7-4350-9e91-a4e143268324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:38.098038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5636e-13e7-4350-9e91-a4e143268324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:38.098320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:38.100867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:38.101294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:38.120317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:38.122943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:38.146239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 869/1748] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:38.148029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:38.231103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:38.242949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5636e-13e7-4350-9e91-a4e143268324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:38.248198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5636e-13e7-4350-9e91-a4e143268324 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:38.283851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 880/1749] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:38.286289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:38.286720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:38.290163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:38.353614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8235c4-9fa4-4f76-ac4e-cb034694a41c service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:38.368372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726e916-862a-4009-83c0-69d99a1e56bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:38.368800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726e916-862a-4009-83c0-69d99a1e56bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:38.369179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726e916-862a-4009-83c0-69d99a1e56bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:38.369530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726e916-862a-4009-83c0-69d99a1e56bf admin admin] PATH_INFO: `/v3/projects/ca816f5b04454fc49899eb67b8d4f4a3/users/57c5f279d69140749dae4c1f3dcec10e/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:38.374475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726e916-862a-4009-83c0-69d99a1e56bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca816f5b04454fc49899eb67b8d4f4a3, user_id=57c5f279d69140749dae4c1f3dcec10e, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:38.378538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726e916-862a-4009-83c0-69d99a1e56bf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:38.393472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 870/1750] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:38 2022] PUT /identity/v3/projects/ca816f5b04454fc49899eb67b8d4f4a3/users/57c5f279d69140749dae4c1f3dcec10e/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:38.418226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] MFA Rules not processed for user `f9e85a31297b4b2aad6eb597ba47d6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:38.418772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:38.419135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:38.419709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:38.420406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:38.556530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] MFA Rules not processed for user `57c5f279d69140749dae4c1f3dcec10e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:38.559244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:38.622480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:38.736572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 881/1751] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:38.744304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:38.893453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 871/1752] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:38.930735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:38.932783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:38.969798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecce79-0314-405e-9cfe-03a7502e5195 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:38.985391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:38.987705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:38.988372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:38.988859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:38.990953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:38.998678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.143766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 882/1753] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:38 2022] POST /identity/v3/projects => generated 385 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:39.149863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:39.180298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78364fe-2dd5-4c6a-b00f-88268111f699 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:39.361556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2dad6-e95f-409d-8841-fd7628dd6af0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:39.365514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:39.366169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:39.366459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:39.369347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:39.374892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:39.375576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:39.375929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:39.376210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:39.376903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:39.379078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.416144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:39.431555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 883/1754] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:39 2022] POST /identity/v3/users => generated 299 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:39.432649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:39.458200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b2e43-b7d9-4bc2-bdff-8a47c29ba163 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:39.467449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:39.468141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:39.468439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:39.468714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:39.469635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:39.471401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.491134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:39.493048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.500322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 884/1755] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:39.514502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:39.529640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdec40-f763-48ac-948c-bc81a61a5292 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:39.544210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:39.545036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:39.545375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:39.545471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] PATH_INFO: `/v3/projects/27ef8220df504f23ab03410e1d1bdfc1/users/c9e19dc66fa04ca88e7c1cfcac5e6ba5/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:39.552579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 872/1756] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:39.580614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ef8220df504f23ab03410e1d1bdfc1, user_id=c9e19dc66fa04ca88e7c1cfcac5e6ba5, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:39.586055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.623736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 885/1757] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:39 2022] PUT /identity/v3/projects/27ef8220df504f23ab03410e1d1bdfc1/users/c9e19dc66fa04ca88e7c1cfcac5e6ba5/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:39.625661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:39.656105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53220-c513-4f2f-9bbd-d33fadac022b service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:39.665766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:39.681754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:39.832927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ba9fe-6b08-430f-a87f-dfdfb9acefc2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:39.839718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:39.840497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:39.841039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:39.841401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] PATH_INFO: `/v3/users/3eb9c8687dfc4847aa4717d0df66f20c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:39.843430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb9c8687dfc4847aa4717d0df66f20c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:39.846209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.890708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:39.891899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:39.892367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:39.892810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:39.897349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:39.901937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:39.972816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb9c8687dfc4847aa4717d0df66f20c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:39.975305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb9c8687dfc4847aa4717d0df66f20c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:39.976653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 873/1758] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:39.977809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:40.018266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb9c8687dfc4847aa4717d0df66f20c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:40.034205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb9c8687dfc4847aa4717d0df66f20c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:40.039674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b6c44-3779-462c-a7fd-0f2466a2bd32 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:40.048022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a82f6ebe-fb2b-460e-8a84-b1e15775da1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb9c8687dfc4847aa4717d0df66f20c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:40.070982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 886/1759] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:39 2022] DELETE /identity/v3/users/3eb9c8687dfc4847aa4717d0df66f20c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:40.073913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1d7f6-2a6d-4c71-9885-d1f442607987 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:40.074209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1d7f6-2a6d-4c71-9885-d1f442607987 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:40.074500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1d7f6-2a6d-4c71-9885-d1f442607987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:40.111795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:40.117258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:40.121458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:40.125092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] PATH_INFO: `/v3/projects/27ef8220df504f23ab03410e1d1bdfc1/users/c9e19dc66fa04ca88e7c1cfcac5e6ba5/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:40.149981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-49b1d7f6-2a6d-4c71-9885-d1f442607987 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:40.151204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-49b1d7f6-2a6d-4c71-9885-d1f442607987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:40.151320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ef8220df504f23ab03410e1d1bdfc1, user_id=c9e19dc66fa04ca88e7c1cfcac5e6ba5, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:40.174975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:40.306161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:40.361847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 874/1760] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:39 2022] PUT /identity/v3/projects/27ef8220df504f23ab03410e1d1bdfc1/users/c9e19dc66fa04ca88e7c1cfcac5e6ba5/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:40.371113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 887/1761] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:40.373006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:40.373519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:40.374182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:40.374182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:40.382417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:40.434926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] MFA Rules not processed for user `c9e19dc66fa04ca88e7c1cfcac5e6ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:40.437355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:40.572036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 888/1762] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:40.572036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:40.573494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:40.733402 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a7b6c-af3c-433e-8ed0-a9c7719a5831 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:40.745370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:40.749245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:40.749490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:40.749829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:40.799287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec94d3-bb74-4f16-8e01-13070b7830b0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:40.818093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:40.818794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:40.819070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:40.819350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] PATH_INFO: `/v3/projects/ee3d21cda75742ef9ee0aceb194da05e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:40.829043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3d21cda75742ef9ee0aceb194da05e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:40.831136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:40.939987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:40.941533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:40.988989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 889/1763] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:40.990092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:40.996425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:41.030127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3d21cda75742ef9ee0aceb194da05e', 'request_id': 'req-53b10806-72af-4969-b831-c45bb1960eef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:41.048541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3d21cda75742ef9ee0aceb194da05e', 'request_id': 'req-53b10806-72af-4969-b831-c45bb1960eef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:41.062455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3d21cda75742ef9ee0aceb194da05e', 'request_id': 'req-53b10806-72af-4969-b831-c45bb1960eef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:41.079811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:41.097784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] The token cache is being invalidate because project ee3d21cda75742ef9ee0aceb194da05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:41.099684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:41.100646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-53b10806-72af-4969-b831-c45bb1960eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:41.105974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 875/1764] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:40 2022] DELETE /identity/v3/projects/ee3d21cda75742ef9ee0aceb194da05e => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:41.111905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310351-788a-4cc6-8c79-7ff77958e787 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:41.112830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310351-788a-4cc6-8c79-7ff77958e787 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:41.113215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e310351-788a-4cc6-8c79-7ff77958e787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:41.218222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8e310351-788a-4cc6-8c79-7ff77958e787 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:41.230901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8e310351-788a-4cc6-8c79-7ff77958e787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:41.237307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd73a38-f223-4d06-a658-776a319dd554 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:41.280429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:41.381587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:41.382285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:41.382574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:41.382850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:41.390014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:41.435602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 876/1765] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:41.445421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:41.446221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:41.447057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:41.495072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:41.499415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:41.533557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 877/1766] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:41.538243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:41.570863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0517f0-c8e2-42ce-b470-cc0ef55e95ce None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:41.600840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:41.601617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:41.601917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:41.602242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:41.603077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:41.604034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:41.605169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:41.623597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:41.626089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:41.655026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 878/1767] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:41.657512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:41.691277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a57e6-5683-4cb8-bc05-ecd7ac0cc159 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:41.692701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 890/1768] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:41.742345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:41.747898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:41.748186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:41.748531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:41.755635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:41.757800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:41.770184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:41.771780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:41.807393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5627f-9633-41ee-9e26-72bbc3e3b62b service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:41.844348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 879/1769] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:41 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:41.852302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:41.857935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:41.858663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:41.858961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:41.859244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] PATH_INFO: `/v3/users/21d36a12c6424768a71eff3f151a6def` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:41.860577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d36a12c6424768a71eff3f151a6def)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:41.863625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:42.022308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d36a12c6424768a71eff3f151a6def'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.024066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d36a12c6424768a71eff3f151a6def'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.055056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d36a12c6424768a71eff3f151a6def'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.069073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d36a12c6424768a71eff3f151a6def'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.077618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:42.080821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d36a12c6424768a71eff3f151a6def'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.103669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 891/1770] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:41 2022] DELETE /identity/v3/users/21d36a12c6424768a71eff3f151a6def => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:42.104720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:42.104957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24b18-b9d5-418c-9756-245bf8b35d80 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:42.107587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.331843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88901742-1c35-4a7e-bb44-389d476c4d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:42.333917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88901742-1c35-4a7e-bb44-389d476c4d08 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.334210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88901742-1c35-4a7e-bb44-389d476c4d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.334486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88901742-1c35-4a7e-bb44-389d476c4d08 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.335209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88901742-1c35-4a7e-bb44-389d476c4d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:42.351333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88901742-1c35-4a7e-bb44-389d476c4d08 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:42.359412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb27223-a729-452e-a469-7b0d1f192480 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:42.368844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:42.369961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.370727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.371092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] PATH_INFO: `/v3/users/f9e85a31297b4b2aad6eb597ba47d6a5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.372872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e85a31297b4b2aad6eb597ba47d6a5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:42.377049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:42.433822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 880/1771] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:41 2022] POST /identity/v3/users => generated 292 bytes in 578 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:42.434385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.435348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.437271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.497979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:42.502577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.541259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e85a31297b4b2aad6eb597ba47d6a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.541493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 881/1772] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:42.541631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e85a31297b4b2aad6eb597ba47d6a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.541771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:42.557899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.570895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e85a31297b4b2aad6eb597ba47d6a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.580680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b122f89-802c-403d-9d89-89d03d640c00 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:42.605259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:42.606222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.607136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.609002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.610204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:42.612663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:42.627806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 882/1773] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:42.629351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:42.632299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.639763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:42.667183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e85a31297b4b2aad6eb597ba47d6a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.704570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e85a31297b4b2aad6eb597ba47d6a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:42.727187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 892/1774] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:42 2022] DELETE /identity/v3/users/f9e85a31297b4b2aad6eb597ba47d6a5 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:42.729585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:42.733936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.775909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdea1c-a276-4310-b24b-786d778d488f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:42.876134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:42.877242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.877688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.878155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.881364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.889411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2156930-ad58-41eb-96a2-77c99e53983c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:42.895937 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a3e8c-6649-4c22-9f90-da8938b0e72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:42.896853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a3e8c-6649-4c22-9f90-da8938b0e72e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.896853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a3e8c-6649-4c22-9f90-da8938b0e72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.896853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a3e8c-6649-4c22-9f90-da8938b0e72e admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.896853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a3e8c-6649-4c22-9f90-da8938b0e72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:42.896853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a3e8c-6649-4c22-9f90-da8938b0e72e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:42.912228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:42.916535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:42.919910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 893/1775] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:42.937227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.937934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.940191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.947807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 883/1776] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:42.951489 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:42.971210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:42.972538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae997fd-1a89-4a79-9b2b-b9774498226c service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:42.974547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:42.980242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b211597-3e11-4b63-b601-11613656cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:42.982263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211597-3e11-4b63-b601-11613656cbfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:42.983177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211597-3e11-4b63-b601-11613656cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:42.983665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211597-3e11-4b63-b601-11613656cbfd admin admin] PATH_INFO: `/v3/projects/af6a2cf1cc0e44f0a7a1d8ea0974053c/users/32e2d7fba8f3492c8eca3a5139ba5002/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:42.988755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 894/1777] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:43.014533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b211597-3e11-4b63-b601-11613656cbfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6a2cf1cc0e44f0a7a1d8ea0974053c, user_id=32e2d7fba8f3492c8eca3a5139ba5002, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:43.018843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b211597-3e11-4b63-b601-11613656cbfd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:43.023723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.025183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:43.039890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:43.041104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 884/1778] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:42 2022] PUT /identity/v3/projects/af6a2cf1cc0e44f0a7a1d8ea0974053c/users/32e2d7fba8f3492c8eca3a5139ba5002/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:43.055131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bccf-27ea-42c2-bbb5-7dd2cafd50a2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:43.062024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:43.062861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:43.063373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:43.146613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] MFA Rules not processed for user `32e2d7fba8f3492c8eca3a5139ba5002`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:43.149820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:43.156234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:43.156888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:43.157177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:43.157447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:43.318342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 885/1779] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:43.324384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.352822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:43.354321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:43.371764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:43.393630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 895/1780] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:43.433395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234814c-dd72-42aa-af93-3898ea9d408b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:43.449352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:43.450687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:43.451144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:43.451895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:43.453119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:43.455967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:43.512867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 886/1781] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:43 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:43.516678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.532463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e935-d096-432f-a1e1-88b423f18458 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:43.660866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:43.664058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:43.664378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:43.664669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:43.665438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:43.680282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:43.761363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 896/1782] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:43 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:43.766087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.784538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dceb0-1987-4231-b4b7-ac80b3bdd964 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:43.805776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:43.808268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:43.809075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:43.809677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:43.811216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:43.815138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:43.857593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 887/1783] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:43.862230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.877760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173ff7-2980-403e-bccd-4310bba65b07 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:43.883641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:43.884516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:43.884799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:43.885077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] PATH_INFO: `/v3/projects/0ac91dad79f04f4fa061c395bbd18fb8/users/d84e5de7bcec4938bd1d7b5e4f996baa/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:43.909753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac91dad79f04f4fa061c395bbd18fb8, user_id=d84e5de7bcec4938bd1d7b5e4f996baa, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:43.914872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:43.940998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 897/1784] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:43 2022] PUT /identity/v3/projects/0ac91dad79f04f4fa061c395bbd18fb8/users/d84e5de7bcec4938bd1d7b5e4f996baa/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:43.946986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.969201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0156038-6106-4fb8-a429-1e6b60e22dc2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:44.080274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2488b-aefb-45c1-88b9-fcc8ca71d83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:44.081134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2488b-aefb-45c1-88b9-fcc8ca71d83c admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.081515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2488b-aefb-45c1-88b9-fcc8ca71d83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.081904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2488b-aefb-45c1-88b9-fcc8ca71d83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.084063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2488b-aefb-45c1-88b9-fcc8ca71d83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:44.086958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2488b-aefb-45c1-88b9-fcc8ca71d83c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:44.104867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 888/1785] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:44.109797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:44.137466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf3b20-a980-425a-bd76-03c46e21cd7c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:44.151243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:44.159950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.159950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.159950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] PATH_INFO: `/v3/projects/0ac91dad79f04f4fa061c395bbd18fb8/users/d84e5de7bcec4938bd1d7b5e4f996baa/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.173212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac91dad79f04f4fa061c395bbd18fb8, user_id=d84e5de7bcec4938bd1d7b5e4f996baa, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:44.189727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:44.232883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 898/1786] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:44 2022] PUT /identity/v3/projects/0ac91dad79f04f4fa061c395bbd18fb8/users/d84e5de7bcec4938bd1d7b5e4f996baa/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:44.245905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.246679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.247156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.266014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:44.268035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:44.286141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:44.301882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] MFA Rules not processed for user `d84e5de7bcec4938bd1d7b5e4f996baa`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:44.305140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:44.525014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 889/1787] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:44.536205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:44.538447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:44.600987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:44.630741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d351-43c9-4636-96d3-4eca5e7b3760 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:44.673850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:44.679849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.681868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.684303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] PATH_INFO: `/v3/projects/c73be4b269834dc68b46600f1e34d957` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.688832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c73be4b269834dc68b46600f1e34d957)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:44.692631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681d99a-bfea-458e-bb11-96cecf73d776 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:44.694888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:44.711933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490ef52-1c90-4672-8456-94fa8295b0e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:44.712592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ef52-1c90-4672-8456-94fa8295b0e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.712882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ef52-1c90-4672-8456-94fa8295b0e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.713167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490ef52-1c90-4672-8456-94fa8295b0e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.749575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490ef52-1c90-4672-8456-94fa8295b0e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:44.753522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490ef52-1c90-4672-8456-94fa8295b0e5 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:44.801903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 890/1788] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:44.805898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130a2df-0b67-4333-90ec-06788e03fef9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.809087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130a2df-0b67-4333-90ec-06788e03fef9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.810473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130a2df-0b67-4333-90ec-06788e03fef9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.815487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 891/1789] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:20:44 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:20:44.820512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130a2df-0b67-4333-90ec-06788e03fef9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:44.827159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7130a2df-0b67-4333-90ec-06788e03fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:44.857175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c73be4b269834dc68b46600f1e34d957', 'request_id': 'req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:44.870186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c73be4b269834dc68b46600f1e34d957', 'request_id': 'req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:44.899344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c73be4b269834dc68b46600f1e34d957', 'request_id': 'req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:44.943604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] The token cache is being invalidate because project c73be4b269834dc68b46600f1e34d957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:44.943939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:44.959490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:44.959490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 899/1790] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:44 2022] DELETE /identity/v3/projects/c73be4b269834dc68b46600f1e34d957 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:44.962988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:44.965463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:45.045177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130a2df-0b67-4333-90ec-06788e03fef9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:45.050682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:45.051485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:45.051891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:45.052503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] PATH_INFO: `/v3/projects/24826d47eef7427083346628ec8de1d5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:45.053712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24826d47eef7427083346628ec8de1d5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:45.055915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:45.167803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24826d47eef7427083346628ec8de1d5', 'request_id': 'req-aca91933-04f7-45ae-bf43-3e84427414a3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.180712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24826d47eef7427083346628ec8de1d5', 'request_id': 'req-aca91933-04f7-45ae-bf43-3e84427414a3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.182884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7234ec0-ade2-4dbe-8795-ddd6b67c8f59 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:45.198441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24826d47eef7427083346628ec8de1d5', 'request_id': 'req-aca91933-04f7-45ae-bf43-3e84427414a3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.225752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] The token cache is being invalidate because project 24826d47eef7427083346628ec8de1d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:45.226007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.228252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.233300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 892/1791] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:44 2022] DELETE /identity/v3/projects/24826d47eef7427083346628ec8de1d5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:45.237570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:45.239950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:45.311934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:45.369769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:45.371682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:45.372277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:45.374542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] PATH_INFO: `/v3/users/57c5f279d69140749dae4c1f3dcec10e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:45.377822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c5f279d69140749dae4c1f3dcec10e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:45.392852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:45.416652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:45.423830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91933-04f7-45ae-bf43-3e84427414a3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:45.444607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:45.447186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:45.448823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:45.449147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] PATH_INFO: `/v3/projects/bffb60f1c47947158b338e6eb5aa731c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:45.451261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bffb60f1c47947158b338e6eb5aa731c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:45.461372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:45.611757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bffb60f1c47947158b338e6eb5aa731c', 'request_id': 'req-12960c99-21ac-4dc8-b1e7-dfa8a014e246'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.621149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bffb60f1c47947158b338e6eb5aa731c', 'request_id': 'req-12960c99-21ac-4dc8-b1e7-dfa8a014e246'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.621675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c5f279d69140749dae4c1f3dcec10e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.622441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c5f279d69140749dae4c1f3dcec10e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.633826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bffb60f1c47947158b338e6eb5aa731c', 'request_id': 'req-12960c99-21ac-4dc8-b1e7-dfa8a014e246'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.665125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c5f279d69140749dae4c1f3dcec10e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.666216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] The token cache is being invalidate because project bffb60f1c47947158b338e6eb5aa731c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:45.666910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.668298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.674918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 893/1792] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:45 2022] DELETE /identity/v3/projects/bffb60f1c47947158b338e6eb5aa731c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:45.678429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:45.680639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c5f279d69140749dae4c1f3dcec10e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.681670 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:45.710641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-58d0209b-cb2e-493c-a671-e5bbf6c9c98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c5f279d69140749dae4c1f3dcec10e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:45.733199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 900/1793] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:44 2022] DELETE /identity/v3/users/57c5f279d69140749dae4c1f3dcec10e => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:45.734896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:45.735305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:45.735798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:45.786933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:45.791883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:45.808492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960c99-21ac-4dc8-b1e7-dfa8a014e246 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:45.993237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 901/1794] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:20:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:45.994930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:45.997776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.007669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:46.008322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.008607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.008881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.010800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.091188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:46.103668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:46.107963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:46.124400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:46.131023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 894/1795] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:46.133018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:46.135944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a1a54-0df7-4894-818a-0ed069b06178 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:46.136153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.147113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:46.147917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.148271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.148620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] PATH_INFO: `/v3/users/455c72bef439434dba9f5ff0d9e99181` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.149986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=455c72bef439434dba9f5ff0d9e99181)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:46.152540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:46.163539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c150a85-9e0e-4c67-8566-e463eeddd2c0 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:46.168353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:46.169143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.169330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.169746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] PATH_INFO: `/v3/users/c9e19dc66fa04ca88e7c1cfcac5e6ba5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.173778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e19dc66fa04ca88e7c1cfcac5e6ba5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:46.178486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:46.281713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455c72bef439434dba9f5ff0d9e99181'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.284584 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455c72bef439434dba9f5ff0d9e99181'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.305178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e19dc66fa04ca88e7c1cfcac5e6ba5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.306498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e19dc66fa04ca88e7c1cfcac5e6ba5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.314502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455c72bef439434dba9f5ff0d9e99181'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.326000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455c72bef439434dba9f5ff0d9e99181'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.333417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e19dc66fa04ca88e7c1cfcac5e6ba5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.337791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455c72bef439434dba9f5ff0d9e99181'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.354818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e19dc66fa04ca88e7c1cfcac5e6ba5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.355942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e19dc66fa04ca88e7c1cfcac5e6ba5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.367675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 902/1796] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:45 2022] DELETE /identity/v3/users/455c72bef439434dba9f5ff0d9e99181 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:46.369374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:46.371057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.395827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 895/1797] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:46 2022] DELETE /identity/v3/users/c9e19dc66fa04ca88e7c1cfcac5e6ba5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:46.397560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:46.400053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.528422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feee106-437b-455e-8ba8-b93fd1a1f57c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:46.616626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a100e62-6a04-40dd-8096-3387f3a1dee3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:46.626432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:46.626805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.627092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.627720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] PATH_INFO: `/v3/projects/ca816f5b04454fc49899eb67b8d4f4a3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.629881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca816f5b04454fc49899eb67b8d4f4a3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:46.632689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:46.680884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:46.681743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.682174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.682595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.685459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.699878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:46.701864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce015e1-1873-4b90-a229-324d17167a2d service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:46.707582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:46.715984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 903/1798] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:46.747861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca816f5b04454fc49899eb67b8d4f4a3', 'request_id': 'req-253153c7-6d1e-4ea5-8cf5-ef118119617f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.757153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca816f5b04454fc49899eb67b8d4f4a3', 'request_id': 'req-253153c7-6d1e-4ea5-8cf5-ef118119617f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.768322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca816f5b04454fc49899eb67b8d4f4a3', 'request_id': 'req-253153c7-6d1e-4ea5-8cf5-ef118119617f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.777405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:46.784429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] The token cache is being invalidate because project ca816f5b04454fc49899eb67b8d4f4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:46.785143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.788697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:46.790784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 896/1799] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:46 2022] DELETE /identity/v3/projects/ca816f5b04454fc49899eb67b8d4f4a3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:46.797455 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31ff0-698a-4dc5-9681-76ea066b6645 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.798171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31ff0-698a-4dc5-9681-76ea066b6645 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.798592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31ff0-698a-4dc5-9681-76ea066b6645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.828089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-98f31ff0-698a-4dc5-9681-76ea066b6645 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:46.829512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-98f31ff0-698a-4dc5-9681-76ea066b6645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.892059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:46.893707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.928403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 904/1800] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:46.937012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:46.937552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:46.937905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:46.969398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:46.970388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:46.980847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 905/1801] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:46.985172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:46.995076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67c0b2-fca3-405d-a349-2cabb34c9316 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:46.997977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253153c7-6d1e-4ea5-8cf5-ef118119617f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.003140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.004080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.004617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.004745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.005529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.006031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.006433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.006591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] PATH_INFO: `/v3/projects/4c18ed7ee81a46e593d62422a96f3970` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.007291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:47.007700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.008528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c18ed7ee81a46e593d62422a96f3970)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.009489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.011331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.024087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 906/1802] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:47.025707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.028095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.046475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edac86-9343-4b10-829a-be46247ae629 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.054456 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.055203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.055653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.056025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] PATH_INFO: `/v3/projects/27ef8220df504f23ab03410e1d1bdfc1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.057798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ef8220df504f23ab03410e1d1bdfc1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.060526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.097848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c18ed7ee81a46e593d62422a96f3970', 'request_id': 'req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.105706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c18ed7ee81a46e593d62422a96f3970', 'request_id': 'req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.117627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c18ed7ee81a46e593d62422a96f3970', 'request_id': 'req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.127219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ef8220df504f23ab03410e1d1bdfc1', 'request_id': 'req-51b7f00b-4008-4593-9578-69850113b258'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.131618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] The token cache is being invalidate because project 4c18ed7ee81a46e593d62422a96f3970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:47.131804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.132465 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.133736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ef8220df504f23ab03410e1d1bdfc1', 'request_id': 'req-51b7f00b-4008-4593-9578-69850113b258'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.134437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 897/1803] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:46 2022] DELETE /identity/v3/projects/4c18ed7ee81a46e593d62422a96f3970 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:47.138684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.141057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.145759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ef8220df504f23ab03410e1d1bdfc1', 'request_id': 'req-51b7f00b-4008-4593-9578-69850113b258'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.159681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] The token cache is being invalidate because project 27ef8220df504f23ab03410e1d1bdfc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:47.159976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.161252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51b7f00b-4008-4593-9578-69850113b258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:47.163087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 907/1804] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:47 2022] DELETE /identity/v3/projects/27ef8220df504f23ab03410e1d1bdfc1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:47.164508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85222759-6c29-464c-9f74-64f809137b48 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.164790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85222759-6c29-464c-9f74-64f809137b48 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.165061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85222759-6c29-464c-9f74-64f809137b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.187312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-85222759-6c29-464c-9f74-64f809137b48 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:47.188524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-85222759-6c29-464c-9f74-64f809137b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.234539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c09219-e2c8-46eb-8e6c-85f6352b3d18 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.259313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 908/1805] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:47.260651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f8b93-756a-4232-b22e-04ced859f256 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.260938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f8b93-756a-4232-b22e-04ced859f256 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.261213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f8b93-756a-4232-b22e-04ced859f256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.273508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:47.280866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:47.292353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4f6f8b93-756a-4232-b22e-04ced859f256 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:47.294665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4f6f8b93-756a-4232-b22e-04ced859f256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.306530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 909/1806] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:47.308182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fe8f3-c8ed-4f4c-8ed7-a0947deb6916 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.309874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fe8f3-c8ed-4f4c-8ed7-a0947deb6916 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.310255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fe8f3-c8ed-4f4c-8ed7-a0947deb6916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.323552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.324428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.324718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.325019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.326055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.327775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.340231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7a2fe8f3-c8ed-4f4c-8ed7-a0947deb6916 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:47.341467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7a2fe8f3-c8ed-4f4c-8ed7-a0947deb6916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.351789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 910/1807] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:47.353651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.354167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.355368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.371506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 898/1808] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/projects => generated 393 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:47.371506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.382163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:47.384803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.450027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd43d41-9fff-4744-a4ea-598563e082f6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.456840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.458089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.458375 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.458649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.459449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:47.459677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.461424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.482117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 899/1809] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:47.483177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.486990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:47.498029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 911/1810] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:47.498885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.502633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fd979-402c-47e0-b901-ad406e504922 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.508365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.509005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.509292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.509573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.509662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607437-6268-4c55-b999-c2feb64cf2f0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.510305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.512259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.514895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.515822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.516193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.516518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.517245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.519513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.539329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 900/1811] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/users => generated 297 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:47.540251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.555391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303637cd-c061-4387-b701-5eef362d041c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.563111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 912/1812] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:47.563111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.572074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.573569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.573907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.574284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.575655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:47.576012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.579499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.597504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 901/1813] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:47.598998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.612493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe12897-5bc6-43b1-803f-22e218b5f172 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.704666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12cb4f-5005-4134-961e-72fce8c6ba7b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.711168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.711955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.712321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.712521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.712695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.713178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.713467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.713748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.714077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.714471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.716207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.717500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.740949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 913/1814] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:47.742410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.744103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 902/1815] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:47.745061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.756724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a9a7-e72e-4735-afc8-0dec6fcbdbe8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.756924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86538896-5555-48e7-930c-5c870d3c6003 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.762149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.762847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.763136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.763415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.764145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.764270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.765115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.765615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.766086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.766997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.767686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.771128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.781787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 903/1816] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:47.783114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.789838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:47.809325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cf961-7e8f-46cf-b431-a1887127466a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.822019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.822665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.822957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.823294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.824060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.826024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.828156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 914/1817] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:47.829488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.850997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 904/1818] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:47.852030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:47.867277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78327c7-a891-4a2f-9e8c-df6a5b53956f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.904746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:47.947049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2f30-1d5d-4939-8414-8f6166bf2590 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:47.952146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.952793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.953105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.953372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.953477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:47.954070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.954180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:47.954412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:47.954754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] PATH_INFO: `/v3/projects/0ce97ef5bc694b3c82d8b34a4a71f9c8/users/06cbd013e399454abe81ddc87e3427c7/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:47.956102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.982742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce97ef5bc694b3c82d8b34a4a71f9c8, user_id=06cbd013e399454abe81ddc87e3427c7, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:47.986085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:47.992875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 905/1819] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:47 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:47.993443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.014890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 915/1820] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:47 2022] PUT /identity/v3/projects/0ce97ef5bc694b3c82d8b34a4a71f9c8/users/06cbd013e399454abe81ddc87e3427c7/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:48.016318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.021403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dc3c8-ea38-4887-81e8-9b4893b1a069 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.104621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970190a-4687-4a1e-90e0-f9908aa70d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.106006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970190a-4687-4a1e-90e0-f9908aa70d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.106289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970190a-4687-4a1e-90e0-f9908aa70d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.106564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970190a-4687-4a1e-90e0-f9908aa70d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.107282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970190a-4687-4a1e-90e0-f9908aa70d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.109016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970190a-4687-4a1e-90e0-f9908aa70d69 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.124378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 906/1821] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:48.124378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b45e8-a26a-4e9d-bf97-7e806dcdaa2d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.124378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b45e8-a26a-4e9d-bf97-7e806dcdaa2d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.124378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b45e8-a26a-4e9d-bf97-7e806dcdaa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.132908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06f4b0-5e67-40f1-a5ad-520d0ff38c75 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.139854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.140509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.140796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.141076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.141793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.144852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.158528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e14b45e8-a26a-4e9d-bf97-7e806dcdaa2d None None] MFA Rules not processed for user `06cbd013e399454abe81ddc87e3427c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:48.159793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e14b45e8-a26a-4e9d-bf97-7e806dcdaa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:48.160102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 916/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:48.161496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.173656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff8c0-a609-4720-bb9a-1c09effa127c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.180486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.181180 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.181468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.181752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] PATH_INFO: `/v3/projects/b6034980dcf440349c2d5c880ff2ebaf/users/de165c2de32c480887180ffa5509ed6a/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.200971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6034980dcf440349c2d5c880ff2ebaf, user_id=de165c2de32c480887180ffa5509ed6a, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.203764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.232897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 917/1823] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:48 2022] PUT /identity/v3/projects/b6034980dcf440349c2d5c880ff2ebaf/users/de165c2de32c480887180ffa5509ed6a/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:48.232897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.256715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 907/1824] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:48.259327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.259826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.260518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.278554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:48.307508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] MFA Rules not processed for user `de165c2de32c480887180ffa5509ed6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:48.308055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:48.321389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9feb4-3f2c-43e6-88e0-e2b0bc825e97 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.327715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.328401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.328687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.328962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.329680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.331332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.345867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 918/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:48.345867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.356516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4177469-3e35-4c9e-b514-cfbcfb711ee6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.371358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.372052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.372411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.372734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.373924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.375379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.395146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 908/1826] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:48.396543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.408684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd7a59-bdea-4fed-aba3-eff00933ecc6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.414408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 919/1827] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/projects => generated 391 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:48.414408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.429191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:48.495293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.496774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.497500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.498133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] PATH_INFO: `/v3/projects/0c7513050ac241299721b569d03d6eed/users/63a0425cf00a4a6382e08994df8108b5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.514286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6548-2449-4a32-be66-61097a61b07b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.518492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7513050ac241299721b569d03d6eed, user_id=63a0425cf00a4a6382e08994df8108b5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.521862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.531728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378705e-1d6c-4092-93de-917d4c5e28ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.533792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378705e-1d6c-4092-93de-917d4c5e28ea admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.534356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378705e-1d6c-4092-93de-917d4c5e28ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.534905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378705e-1d6c-4092-93de-917d4c5e28ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.536907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378705e-1d6c-4092-93de-917d4c5e28ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.541037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378705e-1d6c-4092-93de-917d4c5e28ea admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.556844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 909/1828] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:48 2022] PUT /identity/v3/projects/0c7513050ac241299721b569d03d6eed/users/63a0425cf00a4a6382e08994df8108b5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:48.556844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.615535 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 920/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:48.616234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.616704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.617202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.684653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5c371-b981-40e8-a027-c7bb4dabfaf8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.707119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] MFA Rules not processed for user `63a0425cf00a4a6382e08994df8108b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:48.709825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:48.735220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:48.784305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.785354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.786417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.786954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.788054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.790675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.820379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 921/1830] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:48.820379 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.820664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 910/1831] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/users => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:48.821850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.835077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c39b35-87cc-40f3-a9d3-edccb40db894 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.836108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6d0dc-c465-430b-8901-6dbaf32acc75 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.844241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.844930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.845215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.845487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.846233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.848590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.848879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.849049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.849049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.849049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.849653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.851692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:48.877611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 911/1832] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:48.878550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.885341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:48.905030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 922/1833] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:48 2022] POST /identity/v3/users => generated 300 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:48.907404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:48.932036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211f539-6f02-4f48-b99e-15f3a12fe5c4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:48.940821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:48.941538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:48.942463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:48.942704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] PATH_INFO: `/v3/projects/69b5b57afdc7421fb2215699a6ec839c/users/639247450d7e4c7a8b8de0e8ee76a481/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:48.984239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b5b57afdc7421fb2215699a6ec839c, user_id=639247450d7e4c7a8b8de0e8ee76a481, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:48.989025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.014954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 923/1834] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:48 2022] PUT /identity/v3/projects/69b5b57afdc7421fb2215699a6ec839c/users/639247450d7e4c7a8b8de0e8ee76a481/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:49.016298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.017602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16df510-9f44-4fe2-b4ca-a2e29c43487f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.088745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.089743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.090220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.090775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.092868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:49.115792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407d02-e1db-4283-b077-20f422b7199a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.121539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.122269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.122459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.122741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.123471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.125119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.137084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 924/1835] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:49.138211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.150489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14575f-acf8-4956-bf94-4c63a31e2283 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.155622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.156263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.156555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.157815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.158575 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.160266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.162162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.163662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.171341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 925/1836] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:49.172222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.173883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 912/1837] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:49.177065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.186939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ad2d-c7a6-498d-84d0-cfda960cd0da admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.189914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f740-a057-4c16-9569-67320a50aa9f service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.192754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.193452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.193926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.194245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] PATH_INFO: `/v3/projects/880ea15d8df049158de0311aa891545b/users/0facdb444cbd4e7081f11aec652d43cb/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.201027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.202258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.202757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.203239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] PATH_INFO: `/v3/projects/69b5b57afdc7421fb2215699a6ec839c/users/639247450d7e4c7a8b8de0e8ee76a481/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.207531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b5b57afdc7421fb2215699a6ec839c, user_id=639247450d7e4c7a8b8de0e8ee76a481, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.213653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.215231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=880ea15d8df049158de0311aa891545b, user_id=0facdb444cbd4e7081f11aec652d43cb, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.218270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.232721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 913/1838] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:49 2022] PUT /identity/v3/projects/69b5b57afdc7421fb2215699a6ec839c/users/639247450d7e4c7a8b8de0e8ee76a481/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:49.233599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.242020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 926/1839] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:49 2022] PUT /identity/v3/projects/880ea15d8df049158de0311aa891545b/users/0facdb444cbd4e7081f11aec652d43cb/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:49.246654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.247638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:49.324432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8473-ed89-430f-bb65-3a59b06e4e98 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.344266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b730-600d-402d-92b2-bbb915d7a06a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.351686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df53abb-4f8d-4e0d-8818-5dc3178675a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.352475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53abb-4f8d-4e0d-8818-5dc3178675a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.352940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53abb-4f8d-4e0d-8818-5dc3178675a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.353220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53abb-4f8d-4e0d-8818-5dc3178675a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.353940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53abb-4f8d-4e0d-8818-5dc3178675a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.355610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53abb-4f8d-4e0d-8818-5dc3178675a8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.368073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 927/1840] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:49.369630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.370063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.370183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.405513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] MFA Rules not processed for user `639247450d7e4c7a8b8de0e8ee76a481`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:49.405840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.406711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.406839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:49.409643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.409643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] PATH_INFO: `/v3/users/63a0425cf00a4a6382e08994df8108b5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.410091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a0425cf00a4a6382e08994df8108b5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.414195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.513731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a0425cf00a4a6382e08994df8108b5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:49.516149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a0425cf00a4a6382e08994df8108b5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:49.539005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 928/1841] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:49.540670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.549256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a0425cf00a4a6382e08994df8108b5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:49.567444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a0425cf00a4a6382e08994df8108b5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:49.568379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a0425cf00a4a6382e08994df8108b5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:49.587571 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b19c3-8c9f-4696-b20f-596a6a418fbe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.591580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 914/1842] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:49 2022] DELETE /identity/v3/users/63a0425cf00a4a6382e08994df8108b5 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:49.591580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.597633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:49.643025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:49.649158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:49.695848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b5edd-ffca-40ce-838d-481ff5f87b4c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.700265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6602d68-e5f5-4288-9c22-501ee5768b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.701174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6602d68-e5f5-4288-9c22-501ee5768b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.701661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6602d68-e5f5-4288-9c22-501ee5768b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.702155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6602d68-e5f5-4288-9c22-501ee5768b03 admin admin] PATH_INFO: `/v3/projects/880ea15d8df049158de0311aa891545b/users/0facdb444cbd4e7081f11aec652d43cb/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.702276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.703216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.703936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.704808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.705656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6602d68-e5f5-4288-9c22-501ee5768b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880ea15d8df049158de0311aa891545b, user_id=0facdb444cbd4e7081f11aec652d43cb, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.708581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:49.711190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6602d68-e5f5-4288-9c22-501ee5768b03 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.754545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 929/1843] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:49 2022] PUT /identity/v3/projects/880ea15d8df049158de0311aa891545b/users/0facdb444cbd4e7081f11aec652d43cb/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:49.756889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.757401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.757903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.797965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:49.799773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:49.846401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:49.848631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fb9b3-8059-40a3-b51c-5852750e9c4b service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:49.860512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 915/1844] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:49.861821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.862345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.862892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:49.880742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 930/1845] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:49.891184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:49.892757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:49.906148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] MFA Rules not processed for user `0facdb444cbd4e7081f11aec652d43cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:49.907407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:49.987092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0550c-b3d9-42ca-b36b-36e91bd93ffc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:49.993763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:49.994419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:49.995597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:49.995882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.012990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:50.014457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:50.027918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 931/1846] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:50.028868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:50.030273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.047673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 916/1847] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:50.065199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:50.126717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b3025-51b2-4099-9b62-e290092df6da service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:50.132455 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:50.133376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.133658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.133935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.136829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.149325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12532405-858d-4bf3-900d-484e7ad6db83 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:50.156416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:50.157319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.157775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.158174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.172655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:50.174722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:50.178005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:50.194401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 917/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:50.257237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:50.259710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:50.273824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 932/1849] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:50.379611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:50.392622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.420481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbc3a9-7f32-42ee-933b-cd67d7197c0b service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:50.434374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:50.434374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.434374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.434374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] PATH_INFO: `/v3/projects/0c7513050ac241299721b569d03d6eed` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.436306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7513050ac241299721b569d03d6eed)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:50.440881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:50.444031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:50.532383 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7513050ac241299721b569d03d6eed', 'request_id': 'req-47a5b2c1-26d6-4454-8648-fcbf046db59d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.542553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7513050ac241299721b569d03d6eed', 'request_id': 'req-47a5b2c1-26d6-4454-8648-fcbf046db59d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.559153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7513050ac241299721b569d03d6eed', 'request_id': 'req-47a5b2c1-26d6-4454-8648-fcbf046db59d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.604532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:50.606215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.606702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] The token cache is being invalidate because project 0c7513050ac241299721b569d03d6eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:50.607009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.610055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47a5b2c1-26d6-4454-8648-fcbf046db59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.610055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 918/1850] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:50 2022] DELETE /identity/v3/projects/0c7513050ac241299721b569d03d6eed => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:50.616458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c6884-e33d-4364-8668-58ff83208ab9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.616801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c6884-e33d-4364-8668-58ff83208ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.617035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c6884-e33d-4364-8668-58ff83208ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.627101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f41cba-fa26-4267-831b-2c3c6a125d8a service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:50.643649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f63c6884-e33d-4364-8668-58ff83208ab9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:50.643649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f63c6884-e33d-4364-8668-58ff83208ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.751506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 919/1851] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:50.751506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.751506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.751506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.763975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:50.764800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.765170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.765478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] PATH_INFO: `/v3/users/de165c2de32c480887180ffa5509ed6a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.766955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de165c2de32c480887180ffa5509ed6a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:50.772526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:50.820000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:50.824909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.848041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 920/1852] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:50.856370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:50.870789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0886a1-d209-49d2-a9de-54a553782f1f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:50.874084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de165c2de32c480887180ffa5509ed6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.875343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de165c2de32c480887180ffa5509ed6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.883896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:50.884621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.884914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.885194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.886044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:50.886355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:50.888911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:50.893622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de165c2de32c480887180ffa5509ed6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.904592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de165c2de32c480887180ffa5509ed6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.922411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fe201a1f-0af0-432a-948a-5aaa19a5435b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de165c2de32c480887180ffa5509ed6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:50.926821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 921/1853] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:50.928392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:50.932263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:50.938170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 933/1854] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:50 2022] DELETE /identity/v3/users/de165c2de32c480887180ffa5509ed6a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:50.945673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:50.945970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:50.946250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:50.963487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e4f6a-1e57-4936-a7e3-869922a4d8d4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:50.975149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:50.976556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:51.043282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.043957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.044244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.044520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.045851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.048232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.048909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:51.075121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:51.088301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 934/1855] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:51.095347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 922/1856] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:50 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:51.098873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.100341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:51.100740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.168042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053c3c7-d9da-4144-90b6-0e36caa342f4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.187926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649fd44-b4af-4bc5-8714-f9ac4d4f6c55 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.194039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.194713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.194998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.195277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.196195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.198173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.231092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 923/1857] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:51 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:51.236104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.241225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.241861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.242143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.242418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.252462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.252855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4d68-32cf-46cc-8aae-7abdfbce8cda admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.253930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.259377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.260287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.260584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.260891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.261738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.264700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.265871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 935/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:51.287632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 924/1859] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:51.292910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.308644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d6d8e-2b90-4119-9448-8c28fb3bbb5b service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.315157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7704f-ffe1-46fe-b84d-5f27bdeb9da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.315836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7704f-ffe1-46fe-b84d-5f27bdeb9da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.316122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7704f-ffe1-46fe-b84d-5f27bdeb9da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.316411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7704f-ffe1-46fe-b84d-5f27bdeb9da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.317703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7704f-ffe1-46fe-b84d-5f27bdeb9da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.319367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7704f-ffe1-46fe-b84d-5f27bdeb9da3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.332046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 936/1860] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:51.337916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.357573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec91866-b91c-4c44-8037-ac6062e41821 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.372832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.374021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.374535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.375029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] PATH_INFO: `/v3/projects/1498253edc5c4b488e07a0a1555ce05d/users/b0d511aa95a94d998b447189a53a67d5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.399771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1498253edc5c4b488e07a0a1555ce05d, user_id=b0d511aa95a94d998b447189a53a67d5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.403696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.419988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 925/1861] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:51 2022] PUT /identity/v3/projects/1498253edc5c4b488e07a0a1555ce05d/users/b0d511aa95a94d998b447189a53a67d5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:51.431738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.431997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.432349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.464904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] MFA Rules not processed for user `b0d511aa95a94d998b447189a53a67d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:51.466171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:51.574331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 937/1862] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:51.591781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.594084 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:51.708637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591ff5b-3936-4284-b82f-68f2e1f76239 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.710422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:51.712376 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:51.717524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.720353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.721296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.722284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.741208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.744678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.758225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:51.760529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 926/1863] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:20:51.834239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc484d78-2870-4199-bc96-f65deffe4a40 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:51.840898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:51.842112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:51.842486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:51.843178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] PATH_INFO: `/v3/projects/b6034980dcf440349c2d5c880ff2ebaf` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:51.845734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6034980dcf440349c2d5c880ff2ebaf)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:51.850477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:51.953299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6034980dcf440349c2d5c880ff2ebaf', 'request_id': 'req-1831966b-5e45-45db-9292-3d9e9943ed0d'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:51.960254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6034980dcf440349c2d5c880ff2ebaf', 'request_id': 'req-1831966b-5e45-45db-9292-3d9e9943ed0d'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:51.969349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6034980dcf440349c2d5c880ff2ebaf', 'request_id': 'req-1831966b-5e45-45db-9292-3d9e9943ed0d'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:51.984613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] The token cache is being invalidate because project b6034980dcf440349c2d5c880ff2ebaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:51.984957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:51.985689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:51.987725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 938/1864] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:51 2022] DELETE /identity/v3/projects/b6034980dcf440349c2d5c880ff2ebaf => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:51.992212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:52.004553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b597ec-1c2f-474c-a766-ea12d29e4dcd service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:52.082744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:52.126567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:52.127286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:52.127624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:52.128894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] PATH_INFO: `/v3/users/0facdb444cbd4e7081f11aec652d43cb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:52.132825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0facdb444cbd4e7081f11aec652d43cb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:52.143726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:52.284617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0facdb444cbd4e7081f11aec652d43cb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.286441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0facdb444cbd4e7081f11aec652d43cb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.329318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0facdb444cbd4e7081f11aec652d43cb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.349801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0facdb444cbd4e7081f11aec652d43cb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.351070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0facdb444cbd4e7081f11aec652d43cb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.385621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 927/1865] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:51 2022] DELETE /identity/v3/users/0facdb444cbd4e7081f11aec652d43cb => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:52.719753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:52.722820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:52.737274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831966b-5e45-45db-9292-3d9e9943ed0d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:52.782977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:52.785280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:52.854553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:52.855756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:52.856231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:52.856696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] PATH_INFO: `/v3/users/32e2d7fba8f3492c8eca3a5139ba5002` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:52.858271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e2d7fba8f3492c8eca3a5139ba5002)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:52.860529 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:52.898346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e2a5f-710b-49da-9aec-f7e957d78469 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:52.905666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:52.907087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:52.907547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:52.907931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] PATH_INFO: `/v3/projects/880ea15d8df049158de0311aa891545b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:52.909475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880ea15d8df049158de0311aa891545b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:52.912244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:52.943361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e2d7fba8f3492c8eca3a5139ba5002'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.944329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e2d7fba8f3492c8eca3a5139ba5002'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.946318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:52.967945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e2d7fba8f3492c8eca3a5139ba5002'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.983962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e2d7fba8f3492c8eca3a5139ba5002'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.988135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880ea15d8df049158de0311aa891545b', 'request_id': 'req-b265b511-b8f8-4516-9d72-1490da8a8f26'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.996088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5da1dc62-ac0b-431a-bef5-88fa565c38af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e2d7fba8f3492c8eca3a5139ba5002'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:52.999599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880ea15d8df049158de0311aa891545b', 'request_id': 'req-b265b511-b8f8-4516-9d72-1490da8a8f26'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:53.025208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 939/1866] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:52 2022] DELETE /identity/v3/users/32e2d7fba8f3492c8eca3a5139ba5002 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:53.025584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880ea15d8df049158de0311aa891545b', 'request_id': 'req-b265b511-b8f8-4516-9d72-1490da8a8f26'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:53.025761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.025761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.025761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.048135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] The token cache is being invalidate because project 880ea15d8df049158de0311aa891545b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:53.048745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:53.049813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b265b511-b8f8-4516-9d72-1490da8a8f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:53.053441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 928/1867] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:52 2022] DELETE /identity/v3/projects/880ea15d8df049158de0311aa891545b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:53.061780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bcee2-b545-4f2a-8c4b-c17bb058fe23 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.062243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bcee2-b545-4f2a-8c4b-c17bb058fe23 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.062700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bcee2-b545-4f2a-8c4b-c17bb058fe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.074815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:53.076122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:53.083668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:53.130622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8f6bcee2-b545-4f2a-8c4b-c17bb058fe23 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:53.132013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8f6bcee2-b545-4f2a-8c4b-c17bb058fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:53.190835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 940/1868] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:53.213983 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:53.233067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:53.330060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 929/1869] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:53.332234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.332529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.332818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.371367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:53.372881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:53.394277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 930/1870] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:53.401428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:53.425671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81d347-d2c9-48e6-8350-c824cc1d8d72 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:53.431635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ecab-da4d-4c43-9632-3e4cf879b726 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:53.442892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:53.444517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.447825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.448045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:53.448718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.449164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.449353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.449627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.451817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:53.455126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:53.457298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:53.483139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 931/1871] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:53.484619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:53.493167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:53.496151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:53.509745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31392c1-3c39-44c1-af10-5290120115a9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:53.522907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:53.524698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.526722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.527500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.529976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:53.536565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:53.548407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 941/1872] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:53.623130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 932/1873] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:53 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:53.632365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:53.655047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a8638-5a28-46e2-bff0-88af794d170d service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:53.866870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:53.867756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.868170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.868790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.869880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:53.874246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:53.919840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:53.938372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 942/1874] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:53 2022] POST /identity/v3/users => generated 290 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:53.939844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:53.958304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccb49b-7513-4933-8341-087c7b4b844f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:53.976293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:53.977405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:53.977892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:53.978401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:53.979662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:53.984011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.032254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 933/1875] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:54.033652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.083845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a5f21-1a5e-4906-b3b7-a3fc28ad59f9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.104635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.105784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.106299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.106706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.110023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.114119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.157991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 943/1876] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:54.159509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.176527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f51d9f-2ade-4343-ba44-a69aaa048ad1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.214858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb45fe-9695-4b81-aa01-50a868484732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.217135 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb45fe-9695-4b81-aa01-50a868484732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.218853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb45fe-9695-4b81-aa01-50a868484732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.220209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb45fe-9695-4b81-aa01-50a868484732 admin admin] PATH_INFO: `/v3/projects/c6cce27eee734054b0665111f6496ef3/users/1e6cdce1c71c4053b67425d416b3e7b6/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.267612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb45fe-9695-4b81-aa01-50a868484732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cce27eee734054b0665111f6496ef3, user_id=1e6cdce1c71c4053b67425d416b3e7b6, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.275678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb45fe-9695-4b81-aa01-50a868484732 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.301550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 934/1877] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:54 2022] PUT /identity/v3/projects/c6cce27eee734054b0665111f6496ef3/users/1e6cdce1c71c4053b67425d416b3e7b6/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:54.312796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.315494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:54.317433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.318111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.318528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.338298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246813a7-5fda-46cf-9443-518fca99dcb9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.351244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] MFA Rules not processed for user `1e6cdce1c71c4053b67425d416b3e7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:54.353174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:54.453186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 935/1878] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:54.459571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.461758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.462432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.462796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.463077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] PATH_INFO: `/v3/projects/af6a2cf1cc0e44f0a7a1d8ea0974053c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.464434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6a2cf1cc0e44f0a7a1d8ea0974053c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.466810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.481912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100598-1564-40c8-8a24-ad1683245e7b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.487308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.488158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.488479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.488757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.489451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.491684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.516675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:54.548477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6a2cf1cc0e44f0a7a1d8ea0974053c', 'request_id': 'req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.566513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6a2cf1cc0e44f0a7a1d8ea0974053c', 'request_id': 'req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.571853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 936/1879] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:54 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:54.575014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.592406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6a2cf1cc0e44f0a7a1d8ea0974053c', 'request_id': 'req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.618699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] The token cache is being invalidate because project af6a2cf1cc0e44f0a7a1d8ea0974053c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:54.620047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.620972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.623244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 944/1880] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:54 2022] DELETE /identity/v3/projects/af6a2cf1cc0e44f0a7a1d8ea0974053c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:54.627245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.630633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:54.684329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7314c-47bc-4662-b89f-bcd709731863 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.695278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.695567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.695844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.696132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.696830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.698813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.735032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 937/1881] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:54 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:54.737629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.739348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:54.752355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f952-3bb6-4a3a-93e8-3a38a6616dc5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.761358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460e5f4-e644-4d1b-88b7-2bb58f0e5cff admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.763912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.764649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.765232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.765913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] PATH_INFO: `/v3/users/d84e5de7bcec4938bd1d7b5e4f996baa` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.767341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84e5de7bcec4938bd1d7b5e4f996baa)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.770673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.775822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.776577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.776855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.777351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.778127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.779776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.799877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:54.806541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 938/1882] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:54.811832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.826789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1f6b-84cc-4b7a-8583-f3ef04d86452 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:54.836491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:54.837925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:54.838443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:54.838921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] PATH_INFO: `/v3/projects/14f9455254bb4e17a5a12ffba9d337c6/users/74e8dec95d6f46569e0a45722129ea22/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:54.858838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f9455254bb4e17a5a12ffba9d337c6, user_id=74e8dec95d6f46569e0a45722129ea22, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:54.862083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:54.868678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84e5de7bcec4938bd1d7b5e4f996baa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.874267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84e5de7bcec4938bd1d7b5e4f996baa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.890618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 939/1883] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:54 2022] PUT /identity/v3/projects/14f9455254bb4e17a5a12ffba9d337c6/users/74e8dec95d6f46569e0a45722129ea22/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:54.895392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:54.902993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84e5de7bcec4938bd1d7b5e4f996baa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.920855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84e5de7bcec4938bd1d7b5e4f996baa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.934144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84e5de7bcec4938bd1d7b5e4f996baa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:54.945106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 945/1884] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:54 2022] DELETE /identity/v3/users/d84e5de7bcec4938bd1d7b5e4f996baa => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:55.028003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261acd2-ebed-475d-8cb1-eda21e4784fb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.119337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:55.133542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecab5a-1bec-4770-9ea5-15b8a531981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:55.133937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecab5a-1bec-4770-9ea5-15b8a531981f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.134225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecab5a-1bec-4770-9ea5-15b8a531981f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.134579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecab5a-1bec-4770-9ea5-15b8a531981f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.135305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecab5a-1bec-4770-9ea5-15b8a531981f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:55.137372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecab5a-1bec-4770-9ea5-15b8a531981f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:55.171798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 940/1885] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:55.172684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:55.178753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.206708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2211e8-9796-48c2-8649-10c2aa3adf07 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.216405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:55.217732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.218227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.218969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] PATH_INFO: `/v3/projects/14f9455254bb4e17a5a12ffba9d337c6/users/74e8dec95d6f46569e0a45722129ea22/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.225749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f9455254bb4e17a5a12ffba9d337c6, user_id=74e8dec95d6f46569e0a45722129ea22, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:55.229134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:55.249565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 946/1886] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:55 2022] PUT /identity/v3/projects/14f9455254bb4e17a5a12ffba9d337c6/users/74e8dec95d6f46569e0a45722129ea22/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:55.265645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.266058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.266381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.314435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] MFA Rules not processed for user `74e8dec95d6f46569e0a45722129ea22`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:55.315808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.458803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 941/1887] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:55.478607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:55.478607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.540499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:55.541960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.549052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8c924-f9c7-46e3-bf5e-5c56e6eb4f89 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.553760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bd361-0b13-4c34-a8d6-a0bfba72a7f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:55.556308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bd361-0b13-4c34-a8d6-a0bfba72a7f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.556919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bd361-0b13-4c34-a8d6-a0bfba72a7f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.557236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bd361-0b13-4c34-a8d6-a0bfba72a7f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.571131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bd361-0b13-4c34-a8d6-a0bfba72a7f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:55.572717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bd361-0b13-4c34-a8d6-a0bfba72a7f7 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:55.584067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 947/1888] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:55.631262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bf8b0-81cb-4ecb-a44b-42e44da8445b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.636054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:55.636720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.637005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.637289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] PATH_INFO: `/v3/projects/0ac91dad79f04f4fa061c395bbd18fb8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.638482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac91dad79f04f4fa061c395bbd18fb8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:55.640592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:55.708744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac91dad79f04f4fa061c395bbd18fb8', 'request_id': 'req-3267ea0b-8e9e-4218-9345-56f07ab88b63'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:55.717221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac91dad79f04f4fa061c395bbd18fb8', 'request_id': 'req-3267ea0b-8e9e-4218-9345-56f07ab88b63'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:55.727374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac91dad79f04f4fa061c395bbd18fb8', 'request_id': 'req-3267ea0b-8e9e-4218-9345-56f07ab88b63'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:55.742440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] The token cache is being invalidate because project 0ac91dad79f04f4fa061c395bbd18fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:55.742775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:55.743605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3267ea0b-8e9e-4218-9345-56f07ab88b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:55.745107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 942/1889] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:55 2022] DELETE /identity/v3/projects/0ac91dad79f04f4fa061c395bbd18fb8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:55.750700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.751006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.751306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.774408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:55.776077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.864554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 948/1890] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:55.873043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.873496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.873761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.890809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:55.897419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:55.900174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.913895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 943/1891] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:55.919443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:55.936828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dde982-947a-4a64-b80f-a885b6f541c7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.942232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:55.943097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.943231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.943756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.944487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:55.944814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:55.946616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:55.968908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 949/1892] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:55.970604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:55.985558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a80944-620d-4cfe-8803-c9e164c6085f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.999994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.000767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.001252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.001689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.002543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.004856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.051190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 944/1893] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:55 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:56.053136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.068710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49da83-04d7-4b41-bf53-7dd8f5f1e727 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.085832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:56.136728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.138722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.168754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.169840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.170354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.170841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.171993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.175436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.210226 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 950/1894] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:56 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:20:56.215221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.232090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c806a67-6f27-4391-9935-e2ecbe6aea54 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.255028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.255750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.256075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.256900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.257192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.259008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.261983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79c16-c080-4982-9892-5a12ccffe1d3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.270588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.272622 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.273186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.273652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] PATH_INFO: `/v3/users/1e6cdce1c71c4053b67425d416b3e7b6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.275889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6cdce1c71c4053b67425d416b3e7b6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.279766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 951/1895] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:20:56.279766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.283075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.294235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688279d-04e3-48e9-9c3c-8fd0bab1939c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.301271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.302397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.302687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.303033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.304365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.307263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.325215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 952/1896] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:56.330083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.346728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db30f89-1377-4cd2-a064-695f67267d66 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.366666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d59150-3915-4ae5-9aab-1a1d3e20543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.366666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d59150-3915-4ae5-9aab-1a1d3e20543c admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.366984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d59150-3915-4ae5-9aab-1a1d3e20543c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.367169 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d59150-3915-4ae5-9aab-1a1d3e20543c admin admin] PATH_INFO: `/v3/projects/16e2f7d565304b8ca72fb0af9402e1f0/users/51d8665e64884f2b8447e9d958759a22/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.371613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6cdce1c71c4053b67425d416b3e7b6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.372311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6cdce1c71c4053b67425d416b3e7b6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.388566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d59150-3915-4ae5-9aab-1a1d3e20543c admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e2f7d565304b8ca72fb0af9402e1f0, user_id=51d8665e64884f2b8447e9d958759a22, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.391542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d59150-3915-4ae5-9aab-1a1d3e20543c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.392631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6cdce1c71c4053b67425d416b3e7b6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.407260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6cdce1c71c4053b67425d416b3e7b6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.407718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1d7eebcb-2945-4b86-aa3b-4e7626f306e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6cdce1c71c4053b67425d416b3e7b6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.409766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 953/1897] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:56 2022] PUT /identity/v3/projects/16e2f7d565304b8ca72fb0af9402e1f0/users/51d8665e64884f2b8447e9d958759a22/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:56.421513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.421807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.422100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.431386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 945/1898] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:56 2022] DELETE /identity/v3/users/1e6cdce1c71c4053b67425d416b3e7b6 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:56.438438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.439935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.440276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.462343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] MFA Rules not processed for user `51d8665e64884f2b8447e9d958759a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:56.464260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.466146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:56.468237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.560893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:56.577289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 946/1899] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:56.589798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.592137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 954/1900] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:20:56.595936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.607273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.608874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.672276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:56.682707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f855a-75cf-4d20-87ed-4cb1f37e86a1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.688308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.689103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.689515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.689887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.699034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1877bd-4b2d-4303-b03d-702fdb6174ab None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.701071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.702724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.702974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221ce5d-b8b0-4063-98cb-cbb54526d457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.703649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221ce5d-b8b0-4063-98cb-cbb54526d457 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.703936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221ce5d-b8b0-4063-98cb-cbb54526d457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.704212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221ce5d-b8b0-4063-98cb-cbb54526d457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.714206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 955/1901] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:56.715399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221ce5d-b8b0-4063-98cb-cbb54526d457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.716883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221ce5d-b8b0-4063-98cb-cbb54526d457 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.730925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 947/1902] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:56.736138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.737568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.752356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138485-c942-4725-90e1-9cd65036733e service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.758678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.759426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.759867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.760295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] PATH_INFO: `/v3/users/51d8665e64884f2b8447e9d958759a22` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.761977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d8665e64884f2b8447e9d958759a22)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.764996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.820320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d8665e64884f2b8447e9d958759a22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.820905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d8665e64884f2b8447e9d958759a22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.834969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d8665e64884f2b8447e9d958759a22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.843644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d8665e64884f2b8447e9d958759a22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.852660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d8665e64884f2b8447e9d958759a22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:56.859888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 956/1903] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:56 2022] DELETE /identity/v3/users/51d8665e64884f2b8447e9d958759a22 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:56.865018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.865530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.865874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.901139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:56.902531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.977776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 948/1904] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:56.987725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.989958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:57.055647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ec26-18a7-4ef9-9ab3-7968a4aa3a2e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:57.060080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:57.061648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:57.062751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:57.063404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:57.063737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:57.063976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:57.088395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb108-e6e7-433b-9ac4-c8ad6facca4a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:57.088848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:57.092531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:57.100744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:57.101839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:57.102323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:57.102816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] PATH_INFO: `/v3/projects/c6cce27eee734054b0665111f6496ef3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:57.105848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cce27eee734054b0665111f6496ef3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:57.110893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:57.117059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 957/1905] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:57.175013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cce27eee734054b0665111f6496ef3', 'request_id': 'req-2e319a41-0c87-48c6-b260-f62628ffbb04'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.183371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cce27eee734054b0665111f6496ef3', 'request_id': 'req-2e319a41-0c87-48c6-b260-f62628ffbb04'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.195061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cce27eee734054b0665111f6496ef3', 'request_id': 'req-2e319a41-0c87-48c6-b260-f62628ffbb04'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.217306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] The token cache is being invalidate because project c6cce27eee734054b0665111f6496ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:57.218158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.220029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.222779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 949/1906] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:57 2022] DELETE /identity/v3/projects/c6cce27eee734054b0665111f6496ef3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:57.227312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:57.238793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f72a87-436a-45b1-933e-77420656c8f8 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:57.338115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:57.338719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:57.339001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:57.339425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] PATH_INFO: `/v3/users/74e8dec95d6f46569e0a45722129ea22` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:57.340552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e8dec95d6f46569e0a45722129ea22)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:57.342527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:57.349300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:57.420859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e8dec95d6f46569e0a45722129ea22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.421622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e8dec95d6f46569e0a45722129ea22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.437979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e8dec95d6f46569e0a45722129ea22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.452000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e8dec95d6f46569e0a45722129ea22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.463341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3357810e-1877-44e3-9311-e0789a09ed7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e8dec95d6f46569e0a45722129ea22'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:57.473117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 958/1907] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:57 2022] DELETE /identity/v3/users/74e8dec95d6f46569e0a45722129ea22 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:20:57.785885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:57.788929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:57.813169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e319a41-0c87-48c6-b260-f62628ffbb04 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:57.866927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:57.960847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:57.963710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:57.963710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:57.964232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] PATH_INFO: `/v3/projects/16e2f7d565304b8ca72fb0af9402e1f0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:57.967273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e2f7d565304b8ca72fb0af9402e1f0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:57.974976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:58.156158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e2f7d565304b8ca72fb0af9402e1f0', 'request_id': 'req-08c37f55-e910-433c-8bae-aeb958405a6c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.169355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e2f7d565304b8ca72fb0af9402e1f0', 'request_id': 'req-08c37f55-e910-433c-8bae-aeb958405a6c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.190893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e2f7d565304b8ca72fb0af9402e1f0', 'request_id': 'req-08c37f55-e910-433c-8bae-aeb958405a6c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.220895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] The token cache is being invalidate because project 16e2f7d565304b8ca72fb0af9402e1f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:58.221275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.227692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.236764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 950/1908] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:57 2022] DELETE /identity/v3/projects/16e2f7d565304b8ca72fb0af9402e1f0 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:58.246942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070b2f3-7a3e-45af-99a2-c5ab2f225dcc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:58.247547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070b2f3-7a3e-45af-99a2-c5ab2f225dcc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:58.248084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070b2f3-7a3e-45af-99a2-c5ab2f225dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:58.358727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7070b2f3-7a3e-45af-99a2-c5ab2f225dcc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:58.361420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7070b2f3-7a3e-45af-99a2-c5ab2f225dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:58.398160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:58.401398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:58.592696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 959/1909] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:58.593100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c37f55-e910-433c-8bae-aeb958405a6c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:58.593100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:58.593100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:58.593100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:58.593611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] PATH_INFO: `/v3/projects/14f9455254bb4e17a5a12ffba9d337c6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:58.593691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:58.593691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:58.593886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f9455254bb4e17a5a12ffba9d337c6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:58.594030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:58.594170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:58.657337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:58.660792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:58.689808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 960/1910] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:58.689808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:58.691052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:58.713439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f9455254bb4e17a5a12ffba9d337c6', 'request_id': 'req-f8832a52-50b7-4deb-bbc0-0088c1fc6504'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.737695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:58.739591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f9455254bb4e17a5a12ffba9d337c6', 'request_id': 'req-f8832a52-50b7-4deb-bbc0-0088c1fc6504'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.750765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231e0b-631f-41d3-b2e5-9f3e12842a3b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:58.756132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f9455254bb4e17a5a12ffba9d337c6', 'request_id': 'req-f8832a52-50b7-4deb-bbc0-0088c1fc6504'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.776602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] The token cache is being invalidate because project 14f9455254bb4e17a5a12ffba9d337c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:20:58.777075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.777870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.780686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 951/1911] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:20:58 2022] DELETE /identity/v3/projects/14f9455254bb4e17a5a12ffba9d337c6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:58.781944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:58.786447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:58.898816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:58.900061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:58.900644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:58.901218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] PATH_INFO: `/v3/users/06cbd013e399454abe81ddc87e3427c7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:58.903745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=06cbd013e399454abe81ddc87e3427c7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:58.908066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:58.931925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8832a52-50b7-4deb-bbc0-0088c1fc6504 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:58.940953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:58.942603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:58.943603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:58.944925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:58.945429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:58.947100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:58.948810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:58.952204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0aae7-a06a-4687-8aed-a5b2caa218e8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:58.984170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06cbd013e399454abe81ddc87e3427c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.985187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06cbd013e399454abe81ddc87e3427c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:58.985643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 952/1912] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:58.992006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:58.992337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:58.992694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.013651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06cbd013e399454abe81ddc87e3427c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:59.020982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:59.022894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:59.034347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06cbd013e399454abe81ddc87e3427c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:59.034957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 953/1913] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:59.037577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.061238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b835d8-de5b-46ae-9c39-9f726c8414fb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.061238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6234d8-8f41-4a30-b148-d7129f71828c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.061238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6234d8-8f41-4a30-b148-d7129f71828c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.061238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6234d8-8f41-4a30-b148-d7129f71828c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.062296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6234d8-8f41-4a30-b148-d7129f71828c admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.062296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6234d8-8f41-4a30-b148-d7129f71828c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.065964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6234d8-8f41-4a30-b148-d7129f71828c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.084065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fcdfc2ba-002f-4b50-a5b7-58cfae6aba1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06cbd013e399454abe81ddc87e3427c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:20:59.100711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 961/1914] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:20:58 2022] DELETE /identity/v3/users/06cbd013e399454abe81ddc87e3427c7 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:59.102772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.103335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.103960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.124785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 954/1915] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:59.127505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.127996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.128472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.142263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:59.145256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:59.172826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:59.174999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:59.277214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 962/1916] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:59.278994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.281282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:59.304446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff087-a2e3-4eb3-998b-36f4625e2ae4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.313216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.320656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.320656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.320656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.320656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.320656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.341582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 955/1917] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:20:59.343366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.354915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 963/1918] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/users => generated 292 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:59.366181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da8984-8f3d-48d5-b94e-5674949ac44e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.367420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.379604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.381353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.382347 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.383082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.385416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:20:59.386240 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.390696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.399885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1191b-bce7-4242-9fa5-263a7262cd9f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.418271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.418934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.419229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.419509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.420247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.422400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.426170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 956/1919] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:20:59.428452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.432298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:59.435602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 964/1920] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:59.437080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.455708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c093-4fa7-4102-b2f1-498b262bdfb5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.461898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.462795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.463350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.463350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.464117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.467650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.510484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 965/1921] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:59.511202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.521762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:59.543115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190be3d0-16c8-4df2-9947-9c8182e97fec admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.587771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:59.617963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b6c7a-be5c-4217-8e71-68921198f504 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.626435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.627946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.628173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.630137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.630264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.630570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.630724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.631140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.632526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.635741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.651661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.655820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.660254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 966/1922] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:59.661493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.683951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc5316-c54a-4a65-bb92-ca6eacce479b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.685150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 957/1923] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:59.686901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.694575 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.695263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.695583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.696038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.696554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.699354 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.710162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711a1b9-f08c-42e4-bf2c-0821550fa911 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.733317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794221-1634-4368-98db-59c2e8c4fca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.734106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794221-1634-4368-98db-59c2e8c4fca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.737403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794221-1634-4368-98db-59c2e8c4fca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.737821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794221-1634-4368-98db-59c2e8c4fca8 admin admin] PATH_INFO: `/v3/projects/486a69a51709439aad36192aece6133e/users/0f6938e54a8d49a6b00d3a6694d4e57f/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.750607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 967/1924] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:20:59.757451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.779039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca499fc-72ba-42b0-b8e8-467d96f22628 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.782598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794221-1634-4368-98db-59c2e8c4fca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486a69a51709439aad36192aece6133e, user_id=0f6938e54a8d49a6b00d3a6694d4e57f, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.790316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:59.790626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794221-1634-4368-98db-59c2e8c4fca8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.792236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.792746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.793347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.794924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:59.799512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:59.828320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 958/1925] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:20:59 2022] PUT /identity/v3/projects/486a69a51709439aad36192aece6133e/users/0f6938e54a8d49a6b00d3a6694d4e57f/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:20:59.833858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 968/1926] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:20:59.837116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:59.837699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:59.838201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:59.840493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:59.869714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5465379-07bb-463c-8427-8a8af15e115c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:59.888183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:20:59.891331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:00.062877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 959/1927] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:00.068433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:00.069314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.069856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.070719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.071996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:00.073866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:00.078830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:00.081396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:00.115273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 969/1928] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:00.123726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:00.153718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36eee-1df9-4bfe-bd02-bad601a75ea0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:00.167244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80b518-d654-484f-afd3-560b96de8e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:00.168195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80b518-d654-484f-afd3-560b96de8e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.168777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80b518-d654-484f-afd3-560b96de8e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.170998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80b518-d654-484f-afd3-560b96de8e6b admin admin] PATH_INFO: `/v3/projects/ad90b38778fd418e80d5da9b1b26d09a/users/b3f795f2c311494b949ec43e2f1be1f3/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.205595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80b518-d654-484f-afd3-560b96de8e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad90b38778fd418e80d5da9b1b26d09a, user_id=b3f795f2c311494b949ec43e2f1be1f3, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:00.211387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80b518-d654-484f-afd3-560b96de8e6b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:00.241738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1153f-ab5e-4213-9cd5-e32e8f2441d2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:00.246356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 970/1929] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:00 2022] PUT /identity/v3/projects/ad90b38778fd418e80d5da9b1b26d09a/users/b3f795f2c311494b949ec43e2f1be1f3/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:00.247613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514a275-175a-4c11-9ac1-5c68b3a55d2c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.247906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514a275-175a-4c11-9ac1-5c68b3a55d2c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.248187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514a275-175a-4c11-9ac1-5c68b3a55d2c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.249781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 971/1930] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:21:00.257761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dca25-6b92-4593-957e-1cdbab689665 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.260547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dca25-6b92-4593-957e-1cdbab689665 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.261382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dca25-6b92-4593-957e-1cdbab689665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.324395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:00.342325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c50dca25-6b92-4593-957e-1cdbab689665 None None] MFA Rules not processed for user `b3f795f2c311494b949ec43e2f1be1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:00.344175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c50dca25-6b92-4593-957e-1cdbab689665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:00.377199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:00.380141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.380710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.381156 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.466585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 972/1931] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:00.468497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3131a-3381-428d-b643-225657949eac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.469060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3131a-3381-428d-b643-225657949eac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.469538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3131a-3381-428d-b643-225657949eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.497446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2ba3131a-3381-428d-b643-225657949eac None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:00.500651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ba3131a-3381-428d-b643-225657949eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:00.529559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:00.634594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:00.639021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:00.668531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 960/1932] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:00.669932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:00.701416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d50d9-cfad-4682-876a-0014c49df35d service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:00.715582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:00.716302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.718924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.718924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.719287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 973/1933] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:21:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:00.720099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:00.720710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3131a-3381-428d-b643-225657949eac None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:00.721533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:00.726633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ba3131a-3381-428d-b643-225657949eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:00.750620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3131a-3381-428d-b643-225657949eac None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:00.758996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:00.760385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:00.762558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:00.763885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] PATH_INFO: `/v3/projects/0ce97ef5bc694b3c82d8b34a4a71f9c8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:00.765517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce97ef5bc694b3c82d8b34a4a71f9c8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:00.769973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:00.826486 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 961/1934] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:00 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:00.830560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:00.882521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce97ef5bc694b3c82d8b34a4a71f9c8', 'request_id': 'req-f5490366-bdd7-421a-91e5-b88a72dd7ea9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:00.916508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce97ef5bc694b3c82d8b34a4a71f9c8', 'request_id': 'req-f5490366-bdd7-421a-91e5-b88a72dd7ea9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:00.944031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce97ef5bc694b3c82d8b34a4a71f9c8', 'request_id': 'req-f5490366-bdd7-421a-91e5-b88a72dd7ea9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:00.967341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] The token cache is being invalidate because project 0ce97ef5bc694b3c82d8b34a4a71f9c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:00.967672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:00.969229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:00.971390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 974/1935] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:00 2022] DELETE /identity/v3/projects/0ce97ef5bc694b3c82d8b34a4a71f9c8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:00.972177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db336411-f078-4bdf-b30e-75c21889d478 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:00.973020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:00.978993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.086737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.088814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.089133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.089415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.091140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:01.097372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:01.119872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:01.153672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 962/1936] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:00 2022] POST /identity/v3/users => generated 298 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:01.154197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:01.162663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.174829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490366-bdd7-421a-91e5-b88a72dd7ea9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:01.185621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f2a64-503c-454c-8aae-38c94b5d07f7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:01.187346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.188600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.188945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.189231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.193567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.194040 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.195129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.195760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.196328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] PATH_INFO: `/v3/users/639247450d7e4c7a8b8de0e8ee76a481` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.197996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639247450d7e4c7a8b8de0e8ee76a481)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:01.200692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:01.219554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:01.221746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:01.241763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 975/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:01.242823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:01.244512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.266257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f65f24-74f5-4f5d-9010-f817c2531860 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:01.275356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.276532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.277314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.278178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.279479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:01.281934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:01.308798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 976/1938] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:01.312459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:01.314748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.341091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:01.348362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9d114-0c93-4964-8589-506ab9cc5fa6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:01.366932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.367902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.368325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.368708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] PATH_INFO: `/v3/users/b0d511aa95a94d998b447189a53a67d5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.370649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d511aa95a94d998b447189a53a67d5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:01.375730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:01.382073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639247450d7e4c7a8b8de0e8ee76a481'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.383654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639247450d7e4c7a8b8de0e8ee76a481'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.403507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639247450d7e4c7a8b8de0e8ee76a481'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.432323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639247450d7e4c7a8b8de0e8ee76a481'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.433411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639247450d7e4c7a8b8de0e8ee76a481'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.465047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 963/1939] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:01 2022] DELETE /identity/v3/users/639247450d7e4c7a8b8de0e8ee76a481 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:01.465047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:01.468480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.484096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d511aa95a94d998b447189a53a67d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.485771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d511aa95a94d998b447189a53a67d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.519893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d511aa95a94d998b447189a53a67d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.535352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d511aa95a94d998b447189a53a67d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.546471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4bb7375b-09d0-4314-9624-ea14986dcd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d511aa95a94d998b447189a53a67d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:01.555304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 977/1940] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:01 2022] DELETE /identity/v3/users/b0d511aa95a94d998b447189a53a67d5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:01.560118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.560649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.560934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.604566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:01.605835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.623778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bd737-67e4-46ec-a397-4239672bba65 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:01.718745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 978/1941] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:01.731514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:01.734565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.768951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.769694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.769994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.770707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] PATH_INFO: `/v3/projects/71878b32044e4b799bae9feb8056e1fa/users/253d680f249f4c51bd1e4a734775fbd0/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:01.804606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] RBAC: Authorizing `identity:create_grant(project_id=71878b32044e4b799bae9feb8056e1fa, user_id=253d680f249f4c51bd1e4a734775fbd0, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:01.807421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:01.815529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:01.837328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 964/1942] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:01 2022] PUT /identity/v3/projects/71878b32044e4b799bae9feb8056e1fa/users/253d680f249f4c51bd1e4a734775fbd0/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:01.841580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:01.848060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:01.856092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06087946-9e9f-4387-a050-c8f10a49ba05 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:01.911055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:01.945606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2319a-9795-4415-a23a-94ba7c38c682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:01.946430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2319a-9795-4415-a23a-94ba7c38c682 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:01.947562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2319a-9795-4415-a23a-94ba7c38c682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:01.947562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2319a-9795-4415-a23a-94ba7c38c682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.048011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6faa2-4b62-444e-a83a-289d24ab901f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:02.055112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:02.055846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2319a-9795-4415-a23a-94ba7c38c682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.055969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.056124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.056407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.057311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.059387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:02.059548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2319a-9795-4415-a23a-94ba7c38c682 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:02.082306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 965/1943] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:02.086308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:02.089359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 979/1944] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:02.107517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b09c5a-7b25-46a6-a0bd-c79fd283dad1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:02.131249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:02.132876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.133479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.134042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] PATH_INFO: `/v3/projects/71878b32044e4b799bae9feb8056e1fa/users/253d680f249f4c51bd1e4a734775fbd0/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.138008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71878b32044e4b799bae9feb8056e1fa, user_id=253d680f249f4c51bd1e4a734775fbd0, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.142924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:02.165208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 966/1945] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:02 2022] PUT /identity/v3/projects/71878b32044e4b799bae9feb8056e1fa/users/253d680f249f4c51bd1e4a734775fbd0/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:02.183229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53422157-4f3b-4f7b-9903-6e1a24857d09 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.183659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53422157-4f3b-4f7b-9903-6e1a24857d09 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.183947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53422157-4f3b-4f7b-9903-6e1a24857d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.224024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-53422157-4f3b-4f7b-9903-6e1a24857d09 None None] MFA Rules not processed for user `253d680f249f4c51bd1e4a734775fbd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:02.225301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-53422157-4f3b-4f7b-9903-6e1a24857d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.405401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 980/1946] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:02.413600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:02.416487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.455312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.455678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.455910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.530069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] MFA Rules not processed for user `0f6938e54a8d49a6b00d3a6694d4e57f`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:02.534925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.636552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863e61e-192b-466b-be18-ad15b8e63d8e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:02.655234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:02.657243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.657409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.657816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.666688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.767088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:02.781703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 981/1947] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:02.783603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:02.786738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.866515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.868061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:02.896332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 967/1948] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:02.917715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:02.919448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.942360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac982b-edc5-465b-88bb-3909eac2476d service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:02.948164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01101905-77f3-48af-92a6-f75040462ab7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:02.953725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:02.954328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.954614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.954896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] PATH_INFO: `/v3/projects/1498253edc5c4b488e07a0a1555ce05d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.957824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1498253edc5c4b488e07a0a1555ce05d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.959863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:02.961993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:02.962674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.962958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.963234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] PATH_INFO: `/v3/projects/69b5b57afdc7421fb2215699a6ec839c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.964547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b5b57afdc7421fb2215699a6ec839c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.966670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.048698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b5b57afdc7421fb2215699a6ec839c', 'request_id': 'req-e1b23f2a-d12d-467a-ac67-d898f41d438b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.060447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b5b57afdc7421fb2215699a6ec839c', 'request_id': 'req-e1b23f2a-d12d-467a-ac67-d898f41d438b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.075969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:03.078238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b5b57afdc7421fb2215699a6ec839c', 'request_id': 'req-e1b23f2a-d12d-467a-ac67-d898f41d438b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.099995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] The token cache is being invalidate because project 69b5b57afdc7421fb2215699a6ec839c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:03.100782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.101835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.104300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 982/1949] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:02 2022] DELETE /identity/v3/projects/69b5b57afdc7421fb2215699a6ec839c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:03.105879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.108229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:03.116011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1498253edc5c4b488e07a0a1555ce05d', 'request_id': 'req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.132177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1498253edc5c4b488e07a0a1555ce05d', 'request_id': 'req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.146026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1498253edc5c4b488e07a0a1555ce05d', 'request_id': 'req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.163391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] The token cache is being invalidate because project 1498253edc5c4b488e07a0a1555ce05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:03.163832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.164717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cc6d89e7-c55a-4a4f-b1f7-6682b7d7c069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:03.169856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 968/1950] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:02 2022] DELETE /identity/v3/projects/1498253edc5c4b488e07a0a1555ce05d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:03.171521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed87d1-1bbd-4245-84fe-5feeac6e1826 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.172314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed87d1-1bbd-4245-84fe-5feeac6e1826 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.172314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed87d1-1bbd-4245-84fe-5feeac6e1826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.195290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d1ed87d1-1bbd-4245-84fe-5feeac6e1826 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:03.196683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d1ed87d1-1bbd-4245-84fe-5feeac6e1826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:03.208913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b23f2a-d12d-467a-ac67-d898f41d438b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.264845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 969/1951] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:03.269598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:03.271022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449aa861-fc19-4068-8cd7-5c72143ceb54 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.272617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449aa861-fc19-4068-8cd7-5c72143ceb54 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.273230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449aa861-fc19-4068-8cd7-5c72143ceb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.280171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.281457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.281777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.282164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.286933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:03.295853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-449aa861-fc19-4068-8cd7-5c72143ceb54 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:03.297454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-449aa861-fc19-4068-8cd7-5c72143ceb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:03.317268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 970/1952] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:03.317268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7618ba-9f17-4040-b443-2990c0033d8d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.317268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7618ba-9f17-4040-b443-2990c0033d8d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.317268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7618ba-9f17-4040-b443-2990c0033d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.350432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5a7618ba-9f17-4040-b443-2990c0033d8d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:03.353569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5a7618ba-9f17-4040-b443-2990c0033d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:03.371157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 971/1953] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:03.373857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.375306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.375923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.388253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.391568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.411002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 983/1954] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:03.412623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.414448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:03.417210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:03.436150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 972/1955] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:03.436254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5d0ba-2b53-4976-87d4-f510ba34ea68 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.437401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.440951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.442134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.442690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.443227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.444678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:03.445285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.448789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.455924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579ec5-7737-4c9d-82b4-88eeed9c5fc5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.463260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.463956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.464245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.464515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.466587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:03.466907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.470170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.478493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 984/1956] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:03.484792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.490624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 973/1957] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:03.494909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.507420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59a1d7-f97b-4d89-87af-ad43b3c242ca admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.511998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74ac4-e840-410b-be68-a9160d9f4004 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.518386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.520018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.520663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.520949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.521227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.521330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.521799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.521916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.522260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.523450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.523868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.527570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.571691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 974/1958] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:03.588364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.591913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 985/1959] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/projects => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:03.596566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.717052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5338d-6567-4558-b16f-14bd17f137b7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.722155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba88a5-9823-4447-b9aa-49bcde274782 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.727944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.728617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.729467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.729467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.730786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.733131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.739867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.740675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.740951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.741234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.741918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.743915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.754008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:03.775356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 986/1960] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/users => generated 280 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:03.781036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.786867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 975/1961] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:03.791578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.795231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ef802-945f-412f-8355-36167ef1fe41 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.800988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.801791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.802141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.802476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.803254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.804905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.820030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98ef21-a7bd-4d1b-8340-d7a2a3ef1803 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.824010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 987/1962] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:03.827944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.829004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.829715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.830016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.830299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.831191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.833461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.841063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761bc42-919c-41e0-9dea-7e7d70597d7b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.845676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 976/1963] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:03.849681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.852924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.854010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.854499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.855656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.857282 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.861037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.863421 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b676672-65f6-451d-b613-83a720d2e35a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.872331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.873190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.873485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.873756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.874466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.881010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 988/1964] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:03.881364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.883870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.897237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 977/1965] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:03.901208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:03.901657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6a7f-2d77-4edb-87cc-9e3bb77d23ca admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.919608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c1748-4132-48c2-a03a-7f15e160fbba admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:03.919763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838f1f-94e1-489c-bd24-33ec3f568482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.919763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838f1f-94e1-489c-bd24-33ec3f568482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.919943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709cba-e70d-42f0-b418-73bebb4996a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:03.920031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838f1f-94e1-489c-bd24-33ec3f568482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.920116 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709cba-e70d-42f0-b418-73bebb4996a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.920201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838f1f-94e1-489c-bd24-33ec3f568482 admin admin] PATH_INFO: `/v3/projects/177d8c3bbd4e42e3b3f622bb59948040/users/1348541cf8714e0b88a445685d6957d1/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.920364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709cba-e70d-42f0-b418-73bebb4996a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.921863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709cba-e70d-42f0-b418-73bebb4996a9 admin admin] PATH_INFO: `/v3/projects/77a3c505bac3469a8eb78b36758dc1e7/users/11f981b6b49f492ebc8e99791434a6b0/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.945071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af709cba-e70d-42f0-b418-73bebb4996a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a3c505bac3469a8eb78b36758dc1e7, user_id=11f981b6b49f492ebc8e99791434a6b0, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.948817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af709cba-e70d-42f0-b418-73bebb4996a9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.958395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9838f1f-94e1-489c-bd24-33ec3f568482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=177d8c3bbd4e42e3b3f622bb59948040, user_id=1348541cf8714e0b88a445685d6957d1, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:03.965911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9838f1f-94e1-489c-bd24-33ec3f568482 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:03.971502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 978/1966] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:03 2022] PUT /identity/v3/projects/77a3c505bac3469a8eb78b36758dc1e7/users/11f981b6b49f492ebc8e99791434a6b0/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:03.987258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:03.987591 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:03.987876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:03.996746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 989/1967] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:03 2022] PUT /identity/v3/projects/177d8c3bbd4e42e3b3f622bb59948040/users/1348541cf8714e0b88a445685d6957d1/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:04.010369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:04.011304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:04.012311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:04.025856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] MFA Rules not processed for user `11f981b6b49f492ebc8e99791434a6b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:04.027147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:04.044894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:04.083131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] MFA Rules not processed for user `1348541cf8714e0b88a445685d6957d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:04.085506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:04.114043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 979/1968] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:04.127859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:04.130176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:04.194133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 990/1969] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:04 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:04.253102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:04.257305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:04.285216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3690-c84f-48b5-9aa7-590b850a695f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:04.293884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:04.295226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:04.296554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:04.297167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:04.323690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:04.326267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:04.337776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 980/1970] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:04.354273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a7d43-4842-437b-915f-3f8084bdce43 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:04.361318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ad40-3e0b-43b2-b00c-2918c8dfff34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:04.361318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ad40-3e0b-43b2-b00c-2918c8dfff34 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:04.361318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ad40-3e0b-43b2-b00c-2918c8dfff34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:04.361318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ad40-3e0b-43b2-b00c-2918c8dfff34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:04.373067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5ad40-3e0b-43b2-b00c-2918c8dfff34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:04.374184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5ad40-3e0b-43b2-b00c-2918c8dfff34 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:04.385943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 991/1971] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:04 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:04.414634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:04.437925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:04.495436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32009b44-8661-4694-a6d2-4c2bd5726c54 service glance] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:04.500745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:04.501407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:04.501694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:04.501972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] PATH_INFO: `/v3/users/1348541cf8714e0b88a445685d6957d1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:04.503215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1348541cf8714e0b88a445685d6957d1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:04.506173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:04.561309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1348541cf8714e0b88a445685d6957d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:04.562033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1348541cf8714e0b88a445685d6957d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:04.576911 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1348541cf8714e0b88a445685d6957d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:04.584560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1348541cf8714e0b88a445685d6957d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:04.585357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1348541cf8714e0b88a445685d6957d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:04.602797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 981/1972] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:04 2022] DELETE /identity/v3/users/1348541cf8714e0b88a445685d6957d1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:04.607431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:04.607932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:04.608398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:04.629332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:04.631152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:04.712452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 992/1973] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:04.723081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:04.724995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:04.809434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56d82-c82b-40fa-a24c-0c83d92c0f91 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:04.814818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469f011-1de0-4f2f-8cb2-b617bdbba952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:04.815552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469f011-1de0-4f2f-8cb2-b617bdbba952 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:04.815839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469f011-1de0-4f2f-8cb2-b617bdbba952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:04.816660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469f011-1de0-4f2f-8cb2-b617bdbba952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:04.828608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469f011-1de0-4f2f-8cb2-b617bdbba952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:04.830351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469f011-1de0-4f2f-8cb2-b617bdbba952 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:04.843726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 982/1974] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:05.212085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.213576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:05.226734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a9cef-8ab6-407c-b424-c5cbdef3cc4f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.231661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.232338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.232628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.232909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] PATH_INFO: `/v3/projects/177d8c3bbd4e42e3b3f622bb59948040` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.234154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=177d8c3bbd4e42e3b3f622bb59948040)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.236155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.287687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:05.298788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177d8c3bbd4e42e3b3f622bb59948040', 'request_id': 'req-c0ab455e-0812-42cf-b93d-aad18ea44126'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.305734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177d8c3bbd4e42e3b3f622bb59948040', 'request_id': 'req-c0ab455e-0812-42cf-b93d-aad18ea44126'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.315273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177d8c3bbd4e42e3b3f622bb59948040', 'request_id': 'req-c0ab455e-0812-42cf-b93d-aad18ea44126'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.330734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] The token cache is being invalidate because project 177d8c3bbd4e42e3b3f622bb59948040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:05.331107 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.332215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c0ab455e-0812-42cf-b93d-aad18ea44126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.334302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 993/1975] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:05 2022] DELETE /identity/v3/projects/177d8c3bbd4e42e3b3f622bb59948040 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:05.339363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.339752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.340036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.368563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:05.370916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:05.414826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:05.456298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 983/1976] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:05.458395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.459466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.459466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.484856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:05.486110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:05.497239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 994/1977] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:05.501983 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.517976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8f0e-2d9b-48a0-9656-48a25301a2bb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.524314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.525120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.525450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.525758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.526557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:05.526874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.528776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.545006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 984/1978] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:05.549486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.560111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979c1fb-7d8a-4a22-b27d-c671822784da None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.567112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.568093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.568524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.568801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.569530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.571618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.605501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 995/1979] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:05 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:05.610226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.623446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845de23-4abb-49b4-840b-de5b346c5f99 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.692005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.692641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.692919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.693190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.693879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.695931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.719198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 985/1980] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:05 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:05.724397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.735504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72482ce-2179-4e63-a20e-bd03a6213296 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.740582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.741230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.741544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.741847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.742570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.744255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.756503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 996/1981] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:05.760881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.773172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a4d30-9060-4c67-ad21-987ffa742f93 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.778237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e5ed0-f3ef-4a38-8fb6-a6500fd385e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.778906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e5ed0-f3ef-4a38-8fb6-a6500fd385e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.779193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e5ed0-f3ef-4a38-8fb6-a6500fd385e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.779498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e5ed0-f3ef-4a38-8fb6-a6500fd385e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.780221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e5ed0-f3ef-4a38-8fb6-a6500fd385e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.781859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e5ed0-f3ef-4a38-8fb6-a6500fd385e1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.794385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 986/1982] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:05.799094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.810024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b85dcd-a6d5-4e8b-aae4-ed8e8fbfbb65 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.814976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.815722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.816012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.816290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] PATH_INFO: `/v3/projects/3b3a6aa942d84e30a7658a64c7e6147a/users/3d75c6d5ee6a442192a33784283be900/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.834119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3a6aa942d84e30a7658a64c7e6147a, user_id=3d75c6d5ee6a442192a33784283be900, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.836921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.852308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 997/1983] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:05 2022] PUT /identity/v3/projects/3b3a6aa942d84e30a7658a64c7e6147a/users/3d75c6d5ee6a442192a33784283be900/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:05.863102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.863398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.863724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.893189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] MFA Rules not processed for user `3d75c6d5ee6a442192a33784283be900`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:05.894971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:05.968457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 987/1984] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:05.973857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.040576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e69452-3227-4c02-b6ad-f77d06b684b1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.045477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.046112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.046395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.046675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.047359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.049326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.082724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 998/1985] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:05 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:06.088834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.101706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01adf01-c848-4d70-aaf8-7dcece32bcdd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.140330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:06.169637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.170359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.170552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.170836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.171544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.173552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.202418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 988/1986] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:06.204020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.216965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8033dce-ad72-4ec3-9c4e-e2f599478189 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.227511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.227511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.227828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.228120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.228715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.230371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.243509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 999/1987] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:06.247505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.261606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdb63f-540c-44ef-b31d-83e0543fab49 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.268699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f7c66-6b60-42af-aa5c-228dbf9f8d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.270296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f7c66-6b60-42af-aa5c-228dbf9f8d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.270724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f7c66-6b60-42af-aa5c-228dbf9f8d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.271020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f7c66-6b60-42af-aa5c-228dbf9f8d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.271842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f7c66-6b60-42af-aa5c-228dbf9f8d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.273677 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f7c66-6b60-42af-aa5c-228dbf9f8d72 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.291703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 989/1988] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:06.295179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.311686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf7bc3-1a69-475f-bb08-5985b665edd4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.318332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.319381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.319816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.320165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] PATH_INFO: `/v3/projects/44c3b53f79f04f69a986bdde02cae51a/users/eb1cba547a754b14a2fc061c12aaeeaa/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.337469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c3b53f79f04f69a986bdde02cae51a, user_id=eb1cba547a754b14a2fc061c12aaeeaa, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.340311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.356700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1000/1989] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:06 2022] PUT /identity/v3/projects/44c3b53f79f04f69a986bdde02cae51a/users/eb1cba547a754b14a2fc061c12aaeeaa/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:06.370856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.371155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.371439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.402874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] MFA Rules not processed for user `eb1cba547a754b14a2fc061c12aaeeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:06.404746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:06.472327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 990/1990] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:06.476818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.486940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:06.549990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e93fb-0d35-4810-adc0-9300758a5d02 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.556174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.556912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.557195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.557426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.558183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.560264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.598719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1001/1991] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:06.600390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.614713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c8ef7-ce05-4144-a47b-f917de0d2502 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.732556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.733310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.733663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.734036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.734763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.736862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.765238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 991/1992] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:06.776704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.801108 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c9454-7a6f-42a1-af0e-ae4bc5489a84 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.810934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.811748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.812010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.812287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.812988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.815135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.828281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1002/1993] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:06.833016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.850593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137bbd4-4202-4ac0-b778-0b0a58006bbc admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.859931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.860635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.860989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.861269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] PATH_INFO: `/v3/projects/bd17b038f0854799a76cbb7a2cdb5cf5/users/b3ba2a4ccd5b4011a43f4913d73548d6/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.883554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd17b038f0854799a76cbb7a2cdb5cf5, user_id=b3ba2a4ccd5b4011a43f4913d73548d6, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.886710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.914697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 992/1994] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:06 2022] PUT /identity/v3/projects/bd17b038f0854799a76cbb7a2cdb5cf5/users/b3ba2a4ccd5b4011a43f4913d73548d6/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:06.915598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.939564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23f353-8dc3-4d57-b95c-fe0fa16a6a68 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.091724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87078661-5b0f-4462-802d-fc1f03e2049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.092872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078661-5b0f-4462-802d-fc1f03e2049e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.093185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078661-5b0f-4462-802d-fc1f03e2049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.094342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078661-5b0f-4462-802d-fc1f03e2049e admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.095150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87078661-5b0f-4462-802d-fc1f03e2049e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.098358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87078661-5b0f-4462-802d-fc1f03e2049e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.109005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1003/1995] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:07.114757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.126110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ab99-feb2-4379-889a-1fc5f15b3393 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.132603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00dfd6-2dde-410b-befc-cca24b56c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.133282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00dfd6-2dde-410b-befc-cca24b56c7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.133565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00dfd6-2dde-410b-befc-cca24b56c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.133839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00dfd6-2dde-410b-befc-cca24b56c7f5 admin admin] PATH_INFO: `/v3/projects/bd17b038f0854799a76cbb7a2cdb5cf5/users/b3ba2a4ccd5b4011a43f4913d73548d6/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.136284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00dfd6-2dde-410b-befc-cca24b56c7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd17b038f0854799a76cbb7a2cdb5cf5, user_id=b3ba2a4ccd5b4011a43f4913d73548d6, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.141417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00dfd6-2dde-410b-befc-cca24b56c7f5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.144910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:07.156301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 993/1996] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:07 2022] PUT /identity/v3/projects/bd17b038f0854799a76cbb7a2cdb5cf5/users/b3ba2a4ccd5b4011a43f4913d73548d6/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:07.170349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.170724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.171073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.221078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] MFA Rules not processed for user `b3ba2a4ccd5b4011a43f4913d73548d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:07.223506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:07.223627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae3250-0203-42de-a165-178a521f481a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.224214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae3250-0203-42de-a165-178a521f481a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.224752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae3250-0203-42de-a165-178a521f481a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.251262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e5ae3250-0203-42de-a165-178a521f481a None None] MFA Rules not processed for user `0f6938e54a8d49a6b00d3a6694d4e57f`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:07.253668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e5ae3250-0203-42de-a165-178a521f481a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:07.329912 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1004/1997] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:07.347485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.351177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:07.353305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 994/1998] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:07.359719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:07.477413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e02cb6-4b4f-4ad5-8f1c-90b2e1335b17 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.481370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.482010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.482449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.482732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.562863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.564875 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.575630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1005/1999] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:09.362034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae3250-0203-42de-a165-178a521f481a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:09.373963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae3250-0203-42de-a165-178a521f481a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:09.380762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:09.381693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:09.382075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:09.382418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:09.510427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:09.510427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:09.545455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 995/2000] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:10.280884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:10.297432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13da8f8-5d5f-4457-8ee4-3e7eeb852470 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:10.302178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:10.303037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:10.303308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:10.303673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:10.305977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:10.442785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:10.444461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:10.459518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1006/2001] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:10.984948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:11.001814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cfae1-cd8d-4021-bcb4-494d3c964c61 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:11.012918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7823ad6-7e19-45ff-a598-da2bbbc2c797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:11.014966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7823ad6-7e19-45ff-a598-da2bbbc2c797 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:11.015256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7823ad6-7e19-45ff-a598-da2bbbc2c797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:11.015588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7823ad6-7e19-45ff-a598-da2bbbc2c797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:11.050530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7823ad6-7e19-45ff-a598-da2bbbc2c797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:11.055894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7823ad6-7e19-45ff-a598-da2bbbc2c797 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:11.089529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 996/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:12.517701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:12.521904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.613245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:12.638347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0b3ea-3bd5-4fda-9dcf-1b84db1f6808 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:12.651209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:12.651916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:12.652384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:12.652949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] PATH_INFO: `/v3/users/8b0c5b27345c44ea99d6e73e633f8fa3` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:12.654183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0c5b27345c44ea99d6e73e633f8fa3)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:12.667297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:12.774661 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0c5b27345c44ea99d6e73e633f8fa3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.775837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0c5b27345c44ea99d6e73e633f8fa3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.796857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0c5b27345c44ea99d6e73e633f8fa3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.808936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0c5b27345c44ea99d6e73e633f8fa3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.826966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0c5b27345c44ea99d6e73e633f8fa3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.843604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1007/2003] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:12 2022] DELETE /identity/v3/users/8b0c5b27345c44ea99d6e73e633f8fa3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:12.846463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:12.847689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:12.850122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:12.883260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:12.885032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.997624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 997/2004] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:13.048391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.049937 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.180677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc35f-7ff2-4923-bb76-644a6a3f68df admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.184600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.185233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.185521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.185849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.197878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.199646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.211338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1008/2005] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:13.755763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.757949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.799379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757857e4-97f5-4ba0-9398-e99f287bd478 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.823019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.825830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.826370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.826869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] PATH_INFO: `/v3/projects/657664c470214f10bbada00c008ec87d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.832150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=657664c470214f10bbada00c008ec87d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.838526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.893327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.958352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657664c470214f10bbada00c008ec87d', 'request_id': 'req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.967798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657664c470214f10bbada00c008ec87d', 'request_id': 'req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.980493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657664c470214f10bbada00c008ec87d', 'request_id': 'req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.007224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] The token cache is being invalidate because project 657664c470214f10bbada00c008ec87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:14.007996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.009297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.012553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 998/2006] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:13 2022] DELETE /identity/v3/projects/657664c470214f10bbada00c008ec87d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:14.016644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.033085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d05a-2b8b-4340-8cff-0fc598e4fac9 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.235593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.236259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.236544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.236821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] PATH_INFO: `/v3/users/0d0e57877eaa48c982972e0f098d4653` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.240278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0e57877eaa48c982972e0f098d4653)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.242252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.295831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:14.367049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0e57877eaa48c982972e0f098d4653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.367792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0e57877eaa48c982972e0f098d4653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.389383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0e57877eaa48c982972e0f098d4653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.410292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0e57877eaa48c982972e0f098d4653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.423091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ab77c2c-dea6-4bf3-bb2a-f4a2526fe9d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0e57877eaa48c982972e0f098d4653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.452259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1009/2007] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:14 2022] DELETE /identity/v3/users/0d0e57877eaa48c982972e0f098d4653 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:14.892497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.895743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.915660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645cbf-1397-4c18-ba74-1bd5f96ad5ff admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.028971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.030915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.031202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.031696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] PATH_INFO: `/v3/projects/9d3a6a77177a402896b08ae9595c19ae` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.033734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3a6a77177a402896b08ae9595c19ae)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.036951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.146136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3a6a77177a402896b08ae9595c19ae', 'request_id': 'req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.155357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3a6a77177a402896b08ae9595c19ae', 'request_id': 'req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.171993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3a6a77177a402896b08ae9595c19ae', 'request_id': 'req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.198773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] The token cache is being invalidate because project 9d3a6a77177a402896b08ae9595c19ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:15.199082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.200803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a5d5d1eb-cb5a-4c64-8b16-5e246127f519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.204881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 999/2008] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:14 2022] DELETE /identity/v3/projects/9d3a6a77177a402896b08ae9595c19ae => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:15.212840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.213503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.213714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.249813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:15.255420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:15.424032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1010/2009] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:15.435211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.435963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.436458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.498511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:15.500664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:15.526524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1000/2010] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:15.531701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.551052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdefd10-60c6-4491-873d-f1f8c1b3c007 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.587704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.604187 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.604715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.615697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.617163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:15.617901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.620990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.663712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1011/2011] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:15.668019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.678859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e8208-686c-4066-b2cd-0c2da2ac2452 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.688138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.689627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.689840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.690125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.690905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.693091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.730372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1001/2012] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:15 2022] POST /identity/v3/projects => generated 407 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:15.736549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.760613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ea701-39e3-40cc-bec9-5142db73d428 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.884755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.886102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.886570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.887056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.888501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.891328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.928929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.932920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:15.947873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1012/2013] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:15 2022] POST /identity/v3/users => generated 304 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:15.953017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.958498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a8be4-a8b3-4181-8fcf-9371a7fe27bc admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.965256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.965940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.966228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.966508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] PATH_INFO: `/v3/users/3d75c6d5ee6a442192a33784283be900` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.968011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d75c6d5ee6a442192a33784283be900)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.970147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.974944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434ccbb-2e31-423c-8434-1c5ba29eb912 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.977684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:15.997972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.998271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.998665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.999414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.999975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.001943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.010603 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:16.025638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1013/2014] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:16.027088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.041787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b46e5-14f4-42fa-aada-c282d0a15788 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.050188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.051642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.052267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.052814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.053903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.061322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.074242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d75c6d5ee6a442192a33784283be900'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.076681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d75c6d5ee6a442192a33784283be900'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.089304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1014/2015] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:16.096020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.106100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d75c6d5ee6a442192a33784283be900'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.115616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d75c6d5ee6a442192a33784283be900'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.117887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8401e5-2cf7-461b-98ff-cb7fb071fbcc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.123848 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a85a9-ff65-401d-99f9-95be20f7b2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.125061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a85a9-ff65-401d-99f9-95be20f7b2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.125665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a85a9-ff65-401d-99f9-95be20f7b2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.126091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a85a9-ff65-401d-99f9-95be20f7b2b1 admin admin] PATH_INFO: `/v3/projects/bdb26a419f8d4757b093701659887209/users/b5ebe7dc2ba74c8ca096dc0707769442/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.134972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d75c6d5ee6a442192a33784283be900'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.158017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a85a9-ff65-401d-99f9-95be20f7b2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb26a419f8d4757b093701659887209, user_id=b5ebe7dc2ba74c8ca096dc0707769442, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.161413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a85a9-ff65-401d-99f9-95be20f7b2b1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.168785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1002/2016] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:15 2022] DELETE /identity/v3/users/3d75c6d5ee6a442192a33784283be900 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:16.169999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.173109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.185795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1015/2017] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:16 2022] PUT /identity/v3/projects/bdb26a419f8d4757b093701659887209/users/b5ebe7dc2ba74c8ca096dc0707769442/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:16.188789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bb9f2-cbd5-41f2-861c-4a6e9b9190cf None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.189100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bb9f2-cbd5-41f2-861c-4a6e9b9190cf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.189383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bb9f2-cbd5-41f2-861c-4a6e9b9190cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.218136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3a0bb9f2-cbd5-41f2-861c-4a6e9b9190cf None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.219640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3a0bb9f2-cbd5-41f2-861c-4a6e9b9190cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.301087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1341-66ae-4d07-a80f-8b3137c47704 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.338582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1016/2018] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:16.340023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.340326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.340606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.387113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] MFA Rules not processed for user `b5ebe7dc2ba74c8ca096dc0707769442`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.388774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.411632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.412625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.413057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.413505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] PATH_INFO: `/v3/users/b3f795f2c311494b949ec43e2f1be1f3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.414876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f795f2c311494b949ec43e2f1be1f3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.416981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.484545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:16.502150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1017/2019] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:16.503306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.505825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.516159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f795f2c311494b949ec43e2f1be1f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.517095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f795f2c311494b949ec43e2f1be1f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.532370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f795f2c311494b949ec43e2f1be1f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.543760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f795f2c311494b949ec43e2f1be1f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.544909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f795f2c311494b949ec43e2f1be1f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.578276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1003/2020] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:16 2022] DELETE /identity/v3/users/b3f795f2c311494b949ec43e2f1be1f3 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:16.579100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.581433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.586681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f613b-79c9-47b5-abd0-b877353569ca None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.596568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:16.680962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c944d9-bb39-4e27-9a8f-18e5652e6327 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.683838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.684502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.684788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.685059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.689039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.689669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.690144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.690432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.690917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.691642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.693897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.712595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.714338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fb2eb-8144-4d24-a466-197584a7c082 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.726757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1018/2021] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:16.728056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.728416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.728627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.756840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1004/2022] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/projects => generated 405 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:16.759292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.777921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.780153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.924396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0097d-dcdd-4bcb-9c17-b3d2d4abe451 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.929298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1019/2023] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:16.934599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.935358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.935711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.936000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.936871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.939162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.959291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.961227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.984962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1005/2024] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/users => generated 309 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:16.986057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.997160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2b779-05c0-4d37-89ee-b30d2d5ddbbe admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.002735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.003805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.004184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.004904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.006000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.007724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.027857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1006/2025] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:17.031676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.048139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c45475-98bc-4ee1-83c7-68098646f0e1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.050971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ae925-fbbd-4b1c-b8e5-7bc08f10bd0f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.054871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.055600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.056052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.056374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.057679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.058453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.058845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.059134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] PATH_INFO: `/v3/projects/7e11b4432b864a81a4ef869539969363/users/1f7cc6afe2054685bb3a5282f2752a51/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.066972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.080220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.085980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.093744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1020/2026] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:17.100750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e11b4432b864a81a4ef869539969363, user_id=1f7cc6afe2054685bb3a5282f2752a51, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.107881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.135310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1007/2027] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:17 2022] PUT /identity/v3/projects/7e11b4432b864a81a4ef869539969363/users/1f7cc6afe2054685bb3a5282f2752a51/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:17.140308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.153751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d520493-45fa-42cb-9734-51b1475899c3 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.230509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.232645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.233124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.243443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.244305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.244502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.244790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.245505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.247993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.262761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1021/2028] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:17.263743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.276428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9c1c1-0e5c-40e9-a86e-8b5081fa30d1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.282140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbad7a9-bcb0-4a52-9e1f-510e38eec646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.282861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbad7a9-bcb0-4a52-9e1f-510e38eec646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.283145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbad7a9-bcb0-4a52-9e1f-510e38eec646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.283424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbad7a9-bcb0-4a52-9e1f-510e38eec646 admin admin] PATH_INFO: `/v3/projects/7e11b4432b864a81a4ef869539969363/users/1f7cc6afe2054685bb3a5282f2752a51/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.289491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbad7a9-bcb0-4a52-9e1f-510e38eec646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e11b4432b864a81a4ef869539969363, user_id=1f7cc6afe2054685bb3a5282f2752a51, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.292245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbad7a9-bcb0-4a52-9e1f-510e38eec646 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.316906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1022/2029] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:17 2022] PUT /identity/v3/projects/7e11b4432b864a81a4ef869539969363/users/1f7cc6afe2054685bb3a5282f2752a51/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:17.329530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.329931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.330315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.337796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f92d2-43b1-43cb-aa3b-7379159bb52e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.369342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] MFA Rules not processed for user `1f7cc6afe2054685bb3a5282f2752a51`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:17.370593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.412392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.413305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.414112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.414491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] PATH_INFO: `/v3/projects/3b3a6aa942d84e30a7658a64c7e6147a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.416091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3a6aa942d84e30a7658a64c7e6147a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.426227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.475112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1023/2030] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:17.479161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.479161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.499526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857302b-5ae5-481b-8e3f-12a809941350 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.515533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3a6aa942d84e30a7658a64c7e6147a', 'request_id': 'req-96018063-d905-42f5-8537-78ad072d48eb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.522148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3a6aa942d84e30a7658a64c7e6147a', 'request_id': 'req-96018063-d905-42f5-8537-78ad072d48eb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.530492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3a6aa942d84e30a7658a64c7e6147a', 'request_id': 'req-96018063-d905-42f5-8537-78ad072d48eb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.542773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] The token cache is being invalidate because project 3b3a6aa942d84e30a7658a64c7e6147a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:17.543804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.543939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.545432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1008/2031] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/projects/3b3a6aa942d84e30a7658a64c7e6147a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:17.546236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.550517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.573485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.574177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.574454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.574734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] PATH_INFO: `/v3/projects/ad90b38778fd418e80d5da9b1b26d09a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.575932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad90b38778fd418e80d5da9b1b26d09a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.577856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.584204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.616469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96018063-d905-42f5-8537-78ad072d48eb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.630394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad90b38778fd418e80d5da9b1b26d09a', 'request_id': 'req-3e19b11b-d66a-41b1-9c88-c75db4192147'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.636500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad90b38778fd418e80d5da9b1b26d09a', 'request_id': 'req-3e19b11b-d66a-41b1-9c88-c75db4192147'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.645822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad90b38778fd418e80d5da9b1b26d09a', 'request_id': 'req-3e19b11b-d66a-41b1-9c88-c75db4192147'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.648427 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.663565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] The token cache is being invalidate because project ad90b38778fd418e80d5da9b1b26d09a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:17.663678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.664359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.666024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1024/2032] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/projects/ad90b38778fd418e80d5da9b1b26d09a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:17.666785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.668351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.673569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.674267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.674554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.674831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.677157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.741494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19b11b-d66a-41b1-9c88-c75db4192147 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.747203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.748060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.748432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.748718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] PATH_INFO: `/v3/users/eb1cba547a754b14a2fc061c12aaeeaa` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.749977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1cba547a754b14a2fc061c12aaeeaa)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.751975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.752643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.754746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.767563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1009/2033] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:17 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:17.768662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.770794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.789243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6a55c-3c83-414b-85b9-bd8de097e0de service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.794668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.795771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.795771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.796158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] PATH_INFO: `/v3/users/253d680f249f4c51bd1e4a734775fbd0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.797758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253d680f249f4c51bd1e4a734775fbd0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.800552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.833498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1cba547a754b14a2fc061c12aaeeaa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.834286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1cba547a754b14a2fc061c12aaeeaa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.863066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1cba547a754b14a2fc061c12aaeeaa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.874489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1cba547a754b14a2fc061c12aaeeaa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.879405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253d680f249f4c51bd1e4a734775fbd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.880407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253d680f249f4c51bd1e4a734775fbd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.888099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1cba547a754b14a2fc061c12aaeeaa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.895622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1025/2034] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/users/eb1cba547a754b14a2fc061c12aaeeaa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:17.901975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253d680f249f4c51bd1e4a734775fbd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.913924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253d680f249f4c51bd1e4a734775fbd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.914690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253d680f249f4c51bd1e4a734775fbd0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.934192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1010/2035] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/users/253d680f249f4c51bd1e4a734775fbd0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:18.044215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.047317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.155105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025e8a0-006e-4a99-9b47-55d35d55e656 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.212693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:18.260374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.261027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.261278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.261574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.263388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.349457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.351125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.362070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1026/2036] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:18 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:18.376785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.378460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.383850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.388668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:18.389825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.483313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c373cc-95f0-484f-a6ae-2ea942a77294 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.489756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.490525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.490814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.491095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] PATH_INFO: `/v3/projects/71878b32044e4b799bae9feb8056e1fa` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.491857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1424f4-bc5a-4de3-8b42-cf31d47d4be2 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.492338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=71878b32044e4b799bae9feb8056e1fa)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.494544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.498668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.499403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.499787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.500131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] PATH_INFO: `/v3/projects/44c3b53f79f04f69a986bdde02cae51a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.501611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c3b53f79f04f69a986bdde02cae51a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.503791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.571931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c3b53f79f04f69a986bdde02cae51a', 'request_id': 'req-79694dbb-2579-4275-b9b9-aa0b23a98c81'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.577219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71878b32044e4b799bae9feb8056e1fa', 'request_id': 'req-6d538347-f1d8-4312-9ed9-39568247c8fe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.588291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c3b53f79f04f69a986bdde02cae51a', 'request_id': 'req-79694dbb-2579-4275-b9b9-aa0b23a98c81'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.594710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71878b32044e4b799bae9feb8056e1fa', 'request_id': 'req-6d538347-f1d8-4312-9ed9-39568247c8fe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.599995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c3b53f79f04f69a986bdde02cae51a', 'request_id': 'req-79694dbb-2579-4275-b9b9-aa0b23a98c81'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.607087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71878b32044e4b799bae9feb8056e1fa', 'request_id': 'req-6d538347-f1d8-4312-9ed9-39568247c8fe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.620880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] The token cache is being invalidate because project 44c3b53f79f04f69a986bdde02cae51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:18.621369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.622569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.627600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1027/2037] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:18 2022] DELETE /identity/v3/projects/44c3b53f79f04f69a986bdde02cae51a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:18.628943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.630382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.633669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] The token cache is being invalidate because project 71878b32044e4b799bae9feb8056e1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:18.633978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.635817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6d538347-f1d8-4312-9ed9-39568247c8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.639539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1011/2038] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:18 2022] DELETE /identity/v3/projects/71878b32044e4b799bae9feb8056e1fa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:18.653601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159a6a8-c3be-4bf1-bc7d-0fa7c5530953 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.653601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159a6a8-c3be-4bf1-bc7d-0fa7c5530953 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.654089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159a6a8-c3be-4bf1-bc7d-0fa7c5530953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.660097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:18.690105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a159a6a8-c3be-4bf1-bc7d-0fa7c5530953 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:18.693108 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a159a6a8-c3be-4bf1-bc7d-0fa7c5530953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.745126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79694dbb-2579-4275-b9b9-aa0b23a98c81 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.752166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.752522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.752810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.753085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] PATH_INFO: `/v3/users/b3ba2a4ccd5b4011a43f4913d73548d6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.754582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ba2a4ccd5b4011a43f4913d73548d6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.756702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.802035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1012/2039] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:18.809376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.809722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.810012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.827891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ba2a4ccd5b4011a43f4913d73548d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.828558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ba2a4ccd5b4011a43f4913d73548d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.835953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:18.837662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.848256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1013/2040] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:18.853151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.865315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4c064-c231-47f4-8f5c-771981a40a6e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.868697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ba2a4ccd5b4011a43f4913d73548d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.873239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.874031 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.874445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.874834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.875750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:18.876101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.880190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.884981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ba2a4ccd5b4011a43f4913d73548d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.892821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ba2a4ccd5b4011a43f4913d73548d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.895929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1014/2041] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:18.900517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.901279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1028/2042] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:18 2022] DELETE /identity/v3/users/b3ba2a4ccd5b4011a43f4913d73548d6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:18.902586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.917908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d28a2-5952-459b-81e2-81b274974131 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.930059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:18.994903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.995565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.995854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.996126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.996808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.998819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.029737 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1015/2043] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:19.035337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.046162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac711ad-3a1d-42f4-b59f-547ba0c2bd92 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.118881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.119543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.119828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.120112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.120971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.122965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.148536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1029/2044] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:19.149871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.161706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb47cd-ede1-471c-89dc-e4e06fef1fe5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.167495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.168144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.168434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.168714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.169436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.171067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.182664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1016/2045] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:19.186508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.197775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63042-d9e6-4b81-a91a-24c20a1a8bb3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.202985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.203642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.203932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.204210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.204913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.207070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.218399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1030/2046] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:19.223368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.235441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a43c9-3c3d-49ec-b224-fd9b963fd0af admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.236452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.238692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.241138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3fd13-bceb-4281-85fa-8b625f87eaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.241830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3fd13-bceb-4281-85fa-8b625f87eaf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.242118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3fd13-bceb-4281-85fa-8b625f87eaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.242407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3fd13-bceb-4281-85fa-8b625f87eaf0 admin admin] PATH_INFO: `/v3/projects/db5014b6a5f7422ea900d0637585d96f/users/328847f6b46f483eb6ae0eaedcf5db6b/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.255906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5b413-99eb-4bbb-9003-db9b0993f4bf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.262413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.263161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.263548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.263899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] PATH_INFO: `/v3/projects/bd17b038f0854799a76cbb7a2cdb5cf5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.266122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd17b038f0854799a76cbb7a2cdb5cf5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.269254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.272788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3fd13-bceb-4281-85fa-8b625f87eaf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5014b6a5f7422ea900d0637585d96f, user_id=328847f6b46f483eb6ae0eaedcf5db6b, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.276325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3fd13-bceb-4281-85fa-8b625f87eaf0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.284266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:19.295536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1017/2047] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:19 2022] PUT /identity/v3/projects/db5014b6a5f7422ea900d0637585d96f/users/328847f6b46f483eb6ae0eaedcf5db6b/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:19.308033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.308172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.308485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.330954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd17b038f0854799a76cbb7a2cdb5cf5', 'request_id': 'req-70870891-6638-409a-9bd1-97bfebb09207'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:19.338883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd17b038f0854799a76cbb7a2cdb5cf5', 'request_id': 'req-70870891-6638-409a-9bd1-97bfebb09207'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:19.339509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] MFA Rules not processed for user `328847f6b46f483eb6ae0eaedcf5db6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:19.340757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.348936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd17b038f0854799a76cbb7a2cdb5cf5', 'request_id': 'req-70870891-6638-409a-9bd1-97bfebb09207'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:19.364030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] The token cache is being invalidate because project bd17b038f0854799a76cbb7a2cdb5cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:19.364363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:19.365148 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-70870891-6638-409a-9bd1-97bfebb09207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:19.367082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1031/2048] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:19 2022] DELETE /identity/v3/projects/bd17b038f0854799a76cbb7a2cdb5cf5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:19.371897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468b01-ba1e-4eb5-bdbd-68de3879082d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.372270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468b01-ba1e-4eb5-bdbd-68de3879082d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.373255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468b01-ba1e-4eb5-bdbd-68de3879082d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.395930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-31468b01-ba1e-4eb5-bdbd-68de3879082d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:19.397289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-31468b01-ba1e-4eb5-bdbd-68de3879082d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.434042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1018/2049] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:19.450642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.452124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.470403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1032/2050] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:19.478971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.479259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.479588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.512547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:19.514400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.525739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04506e9-d542-47a6-b16b-1698b1bc120e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.534250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.535351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.535952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.536509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.539996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1033/2051] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:19.544483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.553872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.556671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.561280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baef54-01e8-461e-8fae-3070447e8ec7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.568280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:19.568654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.569316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.569601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.569881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.570747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:19.571051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.572771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.581904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1019/2052] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:19.585942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1034/2053] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:19.590325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.602537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59420e-ba37-44e7-b716-ececce609c01 service glance] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.607367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.608024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.608338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.608618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.609302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.611266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.645828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1020/2054] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:19.650962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.664225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696f94-a5c5-4224-aae1-cdeac99da9c2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.736858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.738662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.739130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.739709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.740808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.743632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.785885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1035/2055] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:19.789984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.821454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16c911-1f74-439f-a52a-0dd3d1078b37 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.830541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.831540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.831837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.832182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.832821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.834692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.850562 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1021/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:19.854445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.869339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe7b6-6fea-4ab2-b175-99ecd7684c02 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.881457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade0545-5b3c-46f1-a5ba-65e530be6da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.882195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0545-5b3c-46f1-a5ba-65e530be6da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.882541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0545-5b3c-46f1-a5ba-65e530be6da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.882884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0545-5b3c-46f1-a5ba-65e530be6da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.883710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade0545-5b3c-46f1-a5ba-65e530be6da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.885377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade0545-5b3c-46f1-a5ba-65e530be6da7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.886987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:19.905642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1036/2057] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:19.906231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.921151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6986b3-a102-4c85-ba1f-c7bfa094dac6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.935273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.939746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.940122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.940458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] PATH_INFO: `/v3/projects/608343ef3db04d8f81b4885891858821/users/0b77ee9bb1c844479a14db09fa09c812/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.965038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=608343ef3db04d8f81b4885891858821, user_id=0b77ee9bb1c844479a14db09fa09c812, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.969146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.995516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1022/2058] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:19 2022] PUT /identity/v3/projects/608343ef3db04d8f81b4885891858821/users/0b77ee9bb1c844479a14db09fa09c812/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:20.007066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.008601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.008897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.084514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] MFA Rules not processed for user `0b77ee9bb1c844479a14db09fa09c812`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:20.085783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:20.211340 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1037/2059] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:20.215763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.322651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38131a-d5ae-448b-8f64-2c22e62dce9e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.328645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.329613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.330229 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.330919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.332448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.334977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.393404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1023/2060] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:20.398150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.412940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321fe29-d059-4ab7-8da0-360b06e94aaa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.479752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.480408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.480696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.480975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.481641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.483588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.508693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1038/2061] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:20.512916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.528981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebafc8-e9fd-487d-9d63-8eecc30328d3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.535496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.535496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.535496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.535496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.536175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.538072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.553853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1024/2062] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:20.559520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.570816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a0b3-eb02-4b71-8633-fa66cd273a57 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.582544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b3550-28c7-4426-b368-a2113c4908ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.583208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b3550-28c7-4426-b368-a2113c4908ee admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.583574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b3550-28c7-4426-b368-a2113c4908ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.583852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b3550-28c7-4426-b368-a2113c4908ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.584563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b3550-28c7-4426-b368-a2113c4908ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.586172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b3550-28c7-4426-b368-a2113c4908ee admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.599501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1039/2063] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:20.602379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.617757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc7070-afa8-42b4-bbb5-11218c7933f5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.622927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.623638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.623920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.624198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] PATH_INFO: `/v3/projects/894730d56f6d40f1a4912ae353006c84/users/6c285c0458b84cad8994626118a6706f/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.649397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=894730d56f6d40f1a4912ae353006c84, user_id=6c285c0458b84cad8994626118a6706f, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.653109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.668005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1025/2064] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:20 2022] PUT /identity/v3/projects/894730d56f6d40f1a4912ae353006c84/users/6c285c0458b84cad8994626118a6706f/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:20.678640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.678932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.679232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.708294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] MFA Rules not processed for user `6c285c0458b84cad8994626118a6706f`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:20.709538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:20.764743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:20.776324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1040/2065] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:20.780867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.835098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:20.847683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd63b3e-02c0-479a-b693-3dfce0e7cc6a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.852628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.853242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.853530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.853806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.854475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.856427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.884060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1026/2066] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:20.888133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.898919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea90c2-8007-400b-bd8d-550a50864331 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.958413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.959034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.959319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.959630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.960305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.962359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.996131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1041/2067] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:21.000382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.011584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad987eb3-8f74-46a3-9328-60f4eec2267c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.016243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.016851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.017136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.017415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.018113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.019785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.030590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1027/2068] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:21.034695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.045166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eaf05-428e-4e6d-9b0d-3e83028b961b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.050745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.051276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.051596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.051869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] PATH_INFO: `/v3/projects/372fb575bdaa49d8b42288286108c7bb/users/d4662ac1efc54b04a18f9a113a65e3c9/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.067734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372fb575bdaa49d8b42288286108c7bb, user_id=d4662ac1efc54b04a18f9a113a65e3c9, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.070373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.084264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1042/2069] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:21 2022] PUT /identity/v3/projects/372fb575bdaa49d8b42288286108c7bb/users/d4662ac1efc54b04a18f9a113a65e3c9/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:21.088419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.099708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34226dfe-1b20-4a07-93c6-cb4280342642 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.162990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903b328-61da-4f22-9da5-7c4b13db15d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.163650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903b328-61da-4f22-9da5-7c4b13db15d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.164058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903b328-61da-4f22-9da5-7c4b13db15d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.164349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903b328-61da-4f22-9da5-7c4b13db15d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.165049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903b328-61da-4f22-9da5-7c4b13db15d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.166660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903b328-61da-4f22-9da5-7c4b13db15d6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.177110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1028/2070] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:21.181546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.192320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7e2e-a8b3-4455-b892-3f5dac177110 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.197316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.198360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.198698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.198974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] PATH_INFO: `/v3/projects/372fb575bdaa49d8b42288286108c7bb/users/d4662ac1efc54b04a18f9a113a65e3c9/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.201023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=372fb575bdaa49d8b42288286108c7bb, user_id=d4662ac1efc54b04a18f9a113a65e3c9, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.203727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.214507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1043/2071] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:21 2022] PUT /identity/v3/projects/372fb575bdaa49d8b42288286108c7bb/users/d4662ac1efc54b04a18f9a113a65e3c9/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:21.227894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.228192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.228480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.258064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] MFA Rules not processed for user `d4662ac1efc54b04a18f9a113a65e3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:21.259340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:21.332592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1029/2072] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:21.344164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.345796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:21.406977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4de8f-f038-4626-b1a3-79e3f021ac9c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.411654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.412244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.412528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.412804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.471467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.473828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.478602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:21.484243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1044/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:24.145828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:24.155339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c45663-cc09-4b66-b81d-b6fac8603b6a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:24.159063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:24.160241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:24.160521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:24.160792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:24.228188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:24.245926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:24.247820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:24.260828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1030/2074] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:24.908305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:24.910689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:24.922153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ec5d4-ff78-4916-a8d3-f6856934b5ee service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:24.925720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a96c61-086a-4635-9e62-f4237f61732c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:24.926319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96c61-086a-4635-9e62-f4237f61732c service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:24.926598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96c61-086a-4635-9e62-f4237f61732c service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:24.926870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96c61-086a-4635-9e62-f4237f61732c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:24.936446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a96c61-086a-4635-9e62-f4237f61732c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:24.938333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a96c61-086a-4635-9e62-f4237f61732c service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:24.959034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1045/2075] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:27.177151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.178932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.248917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f78de-ad93-4186-a200-c57821fbc083 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.254912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.255604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.255896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.256181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] PATH_INFO: `/v3/users/b5ebe7dc2ba74c8ca096dc0707769442` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.257484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ebe7dc2ba74c8ca096dc0707769442)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.259437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.312385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ebe7dc2ba74c8ca096dc0707769442'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:27.313074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ebe7dc2ba74c8ca096dc0707769442'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:27.325930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ebe7dc2ba74c8ca096dc0707769442'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:27.333481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ebe7dc2ba74c8ca096dc0707769442'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:27.334322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ebe7dc2ba74c8ca096dc0707769442'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:27.349089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1031/2076] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:27 2022] DELETE /identity/v3/users/b5ebe7dc2ba74c8ca096dc0707769442 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:27.353611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34677630-07b4-44da-bcda-f52a23544eee None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.354448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34677630-07b4-44da-bcda-f52a23544eee None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.354718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34677630-07b4-44da-bcda-f52a23544eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.374646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-34677630-07b4-44da-bcda-f52a23544eee None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:27.376085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-34677630-07b4-44da-bcda-f52a23544eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.488075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1046/2077] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:27.498271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.500286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.663906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb3c52-b24f-483d-83d5-7ecd60586ab8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.672177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.672830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.673118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.673400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.683900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.685551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.698314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1032/2078] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:28.066924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34677630-07b4-44da-bcda-f52a23544eee None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.068700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-34677630-07b4-44da-bcda-f52a23544eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.091606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34677630-07b4-44da-bcda-f52a23544eee None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:28.098307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:28.099057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.099349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.099670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] PATH_INFO: `/v3/projects/bdb26a419f8d4757b093701659887209` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.101073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb26a419f8d4757b093701659887209)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:28.103228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.165821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb26a419f8d4757b093701659887209', 'request_id': 'req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.173534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb26a419f8d4757b093701659887209', 'request_id': 'req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.181055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb26a419f8d4757b093701659887209', 'request_id': 'req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.193723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] The token cache is being invalidate because project bdb26a419f8d4757b093701659887209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:28.193988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.194632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.196195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1047/2079] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:28 2022] DELETE /identity/v3/projects/bdb26a419f8d4757b093701659887209 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:28.200178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.210646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8287d-b9fa-4301-bfaa-3926f232bba1 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:28.214715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:28.281980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:28.282616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.282903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.283223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] PATH_INFO: `/v3/users/1f7cc6afe2054685bb3a5282f2752a51` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.284431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7cc6afe2054685bb3a5282f2752a51)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:28.286978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.341718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7cc6afe2054685bb3a5282f2752a51'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.342399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7cc6afe2054685bb3a5282f2752a51'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.357441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7cc6afe2054685bb3a5282f2752a51'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.366428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7cc6afe2054685bb3a5282f2752a51'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.367359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1c847e2b-de35-4cb0-a304-5bf9006225fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7cc6afe2054685bb3a5282f2752a51'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.381856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1033/2080] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:28 2022] DELETE /identity/v3/users/1f7cc6afe2054685bb3a5282f2752a51 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:28.719933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.722292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.746511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6af4e-09b4-412a-b9e0-0e4c1974485c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:28.828182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:28.828979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.829271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.829557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] PATH_INFO: `/v3/projects/7e11b4432b864a81a4ef869539969363` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.830750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e11b4432b864a81a4ef869539969363)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:28.833469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.849489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:28.887412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e11b4432b864a81a4ef869539969363', 'request_id': 'req-742c063c-ae03-4ffb-95ae-6277aaa20024'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.892883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e11b4432b864a81a4ef869539969363', 'request_id': 'req-742c063c-ae03-4ffb-95ae-6277aaa20024'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.902915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e11b4432b864a81a4ef869539969363', 'request_id': 'req-742c063c-ae03-4ffb-95ae-6277aaa20024'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.915511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] The token cache is being invalidate because project 7e11b4432b864a81a4ef869539969363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:28.915837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.916478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-742c063c-ae03-4ffb-95ae-6277aaa20024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.918551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1048/2081] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:28 2022] DELETE /identity/v3/projects/7e11b4432b864a81a4ef869539969363 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:28.929047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.929468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.929848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.963477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:28.964691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.046498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1034/2082] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:29.058882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226496b-d010-4212-b647-6263b1540923 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.059179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226496b-d010-4212-b647-6263b1540923 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.059481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226496b-d010-4212-b647-6263b1540923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.082536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d226496b-d010-4212-b647-6263b1540923 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:29.083820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d226496b-d010-4212-b647-6263b1540923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.101218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1049/2083] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:29.106054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.116460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ace0b-35a8-4152-b8f7-49db3919ad3b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.124417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.125149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.125498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.125781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.126660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:29.126975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.128664 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.147549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1035/2084] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:29.149961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226496b-d010-4212-b647-6263b1540923 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.158109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.162940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.165829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226496b-d010-4212-b647-6263b1540923 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.172416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.173420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.174005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.174792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.175960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.178072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.179327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e67a9-868b-40ab-8f48-1fc696571573 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.184785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.185530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.185794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.186073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] PATH_INFO: `/v3/users/0b77ee9bb1c844479a14db09fa09c812` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.187267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b77ee9bb1c844479a14db09fa09c812)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.189430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.200251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:29.230178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1050/2085] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/projects => generated 363 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:29.235629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.254898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b77ee9bb1c844479a14db09fa09c812'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.256297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b77ee9bb1c844479a14db09fa09c812'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.272503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b77ee9bb1c844479a14db09fa09c812'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.280843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b77ee9bb1c844479a14db09fa09c812'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.281598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-620f6e14-c2a8-4dc0-a579-ec5de6ace5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b77ee9bb1c844479a14db09fa09c812'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.296303 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1036/2086] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:29 2022] DELETE /identity/v3/users/0b77ee9bb1c844479a14db09fa09c812 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:29.300745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.301041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.301329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.308774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b2f1c-6798-4b41-8f8a-bbc8de5c5179 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.322271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:29.323612 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.376868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.377497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.377780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.378067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.378800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.380753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.386286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1037/2087] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:29.396631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.398066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.403311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1051/2088] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/users => generated 282 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:29.407825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.410634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.422289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439e9d-cf2c-454d-8b2d-08963690e823 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.427570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.428186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.428579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.428863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.429565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.431166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.433386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:29.444000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1052/2089] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:29.447841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.457934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be6b8c-60ab-4841-853d-1ec55d3efe10 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.459761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d794e0-cb2e-4c8b-a7d1-89de61dd568f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.462750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.463360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.463501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876711c-9691-416e-b9ca-3e472754d625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.463673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.463951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.464040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876711c-9691-416e-b9ca-3e472754d625 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.464269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876711c-9691-416e-b9ca-3e472754d625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.464541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876711c-9691-416e-b9ca-3e472754d625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.464645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.466244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.475545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876711c-9691-416e-b9ca-3e472754d625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.477165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876711c-9691-416e-b9ca-3e472754d625 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.477814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1053/2090] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:29.481847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.488434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1038/2091] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:29.494431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67b817-ae42-4177-a1ab-abd60dc0213c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.501523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.502271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.502587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.502867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] PATH_INFO: `/v3/projects/f4854754e7034ba38d2a27c4cb9b1c3e/users/655f0ce7fedb4436ba693052ce78623b/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.523178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4854754e7034ba38d2a27c4cb9b1c3e, user_id=655f0ce7fedb4436ba693052ce78623b, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.525990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.553852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1054/2092] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:29 2022] PUT /identity/v3/projects/f4854754e7034ba38d2a27c4cb9b1c3e/users/655f0ce7fedb4436ba693052ce78623b/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:29.565875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.566852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.567130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.603425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] MFA Rules not processed for user `655f0ce7fedb4436ba693052ce78623b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:29.604678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.623049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:29.691037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1039/2093] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:29.695962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.761984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60b4ae-4412-4f99-a7c9-c1d1d7c1675a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.767099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.767831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.768204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.768382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.769609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.771587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.797118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.799071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.809125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1055/2094] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:29.810317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.830392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb817-215f-4b9e-84b8-141d191f0bdb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.892971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f689939-95e5-4003-86f5-e5c6faee2993 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.902142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.902142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.902142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.902142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.902574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.902574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.904576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.905212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.905487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.905764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] PATH_INFO: `/v3/projects/608343ef3db04d8f81b4885891858821` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.906973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=608343ef3db04d8f81b4885891858821)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.908957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.926253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1056/2095] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:29.931792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.941772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72153354-4548-4619-ade9-dbfd561bfa99 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.946663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.947278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.947595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.947866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.948562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.950157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.961854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1057/2096] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:29.966008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.968253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608343ef3db04d8f81b4885891858821', 'request_id': 'req-552002b7-d7bd-47cb-b651-ac60702f8435'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.977928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db5a6-b551-4253-a6fa-fc59003d07dc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.980851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608343ef3db04d8f81b4885891858821', 'request_id': 'req-552002b7-d7bd-47cb-b651-ac60702f8435'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.991977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608343ef3db04d8f81b4885891858821', 'request_id': 'req-552002b7-d7bd-47cb-b651-ac60702f8435'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.009172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] The token cache is being invalidate because project 608343ef3db04d8f81b4885891858821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:30.009532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.010397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.012364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1040/2097] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:29 2022] DELETE /identity/v3/projects/608343ef3db04d8f81b4885891858821 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:30.016400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.017961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.023769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.076287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.076966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.077273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.077536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] PATH_INFO: `/v3/projects/189f3d72190e4f4ca8fc43e6d587c125/users/b17506bcb3e04b21a4b1cf4a1c0bbe41/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.085251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.093444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=189f3d72190e4f4ca8fc43e6d587c125, user_id=b17506bcb3e04b21a4b1cf4a1c0bbe41, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.096337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.104764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552002b7-d7bd-47cb-b651-ac60702f8435 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.116231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1058/2098] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:29 2022] PUT /identity/v3/projects/189f3d72190e4f4ca8fc43e6d587c125/users/b17506bcb3e04b21a4b1cf4a1c0bbe41/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:30.120832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.123073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.192915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdc22a-ecef-4bf2-851f-864e3fee6f7f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.196443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.197098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.197385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.197666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] PATH_INFO: `/v3/users/6c285c0458b84cad8994626118a6706f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.197776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.198280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.198563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.198962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.199053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c285c0458b84cad8994626118a6706f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.199699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.201048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.201514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.212409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1059/2099] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:30.216488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.229505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8f3e-d86a-4bb6-9cba-bdb0e7268a8b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.235027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73a305-c95f-47f5-8d68-67dd9a51f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.235781 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a305-c95f-47f5-8d68-67dd9a51f728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.236065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a305-c95f-47f5-8d68-67dd9a51f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.236520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a305-c95f-47f5-8d68-67dd9a51f728 admin admin] PATH_INFO: `/v3/projects/189f3d72190e4f4ca8fc43e6d587c125/users/b17506bcb3e04b21a4b1cf4a1c0bbe41/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.238934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73a305-c95f-47f5-8d68-67dd9a51f728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189f3d72190e4f4ca8fc43e6d587c125, user_id=b17506bcb3e04b21a4b1cf4a1c0bbe41, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.242124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73a305-c95f-47f5-8d68-67dd9a51f728 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.254492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c285c0458b84cad8994626118a6706f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.254957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1060/2100] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:30 2022] PUT /identity/v3/projects/189f3d72190e4f4ca8fc43e6d587c125/users/b17506bcb3e04b21a4b1cf4a1c0bbe41/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:30.255502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c285c0458b84cad8994626118a6706f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.265361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.265696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.266244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.271698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c285c0458b84cad8994626118a6706f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.284662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c285c0458b84cad8994626118a6706f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.285554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c285c0458b84cad8994626118a6706f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.300235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] MFA Rules not processed for user `b17506bcb3e04b21a4b1cf4a1c0bbe41`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:30.302561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.311141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1041/2101] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:30 2022] DELETE /identity/v3/users/6c285c0458b84cad8994626118a6706f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:30.396046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1061/2102] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:30.408203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.409801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.494384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcc341-623d-4565-8532-db1215660803 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.501522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.502236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.502722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.503007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.548363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.624882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.627884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.649979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1042/2103] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:30.685573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.687595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.738230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.787167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.798750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e6fa-5445-4556-9ed3-b641dbaf07ba service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.803715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.804044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b85f4-7109-4a1b-b17c-9e2ce2d32622 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.804721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.805015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.805355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.807480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.871602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.872386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.872739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.872949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] PATH_INFO: `/v3/projects/894730d56f6d40f1a4912ae353006c84` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.876231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] RBAC: Authorizing `identity:delete_project(project_id=894730d56f6d40f1a4912ae353006c84)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.876231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.893418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.895073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.913885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1043/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:30.955905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894730d56f6d40f1a4912ae353006c84', 'request_id': 'req-ba2a6f9e-9b23-4372-a788-315054e1b27b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.965277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894730d56f6d40f1a4912ae353006c84', 'request_id': 'req-ba2a6f9e-9b23-4372-a788-315054e1b27b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.977025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894730d56f6d40f1a4912ae353006c84', 'request_id': 'req-ba2a6f9e-9b23-4372-a788-315054e1b27b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.992011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] The token cache is being invalidate because project 894730d56f6d40f1a4912ae353006c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:30.992486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.993246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.995342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1062/2105] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:30 2022] DELETE /identity/v3/projects/894730d56f6d40f1a4912ae353006c84 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:30.999914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.015745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14e8d6-2726-4949-bc60-36d9735e0530 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.098577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.100418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.103481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.104712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] PATH_INFO: `/v3/users/d4662ac1efc54b04a18f9a113a65e3c9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.112325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4662ac1efc54b04a18f9a113a65e3c9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.117202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.210244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4662ac1efc54b04a18f9a113a65e3c9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.211198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4662ac1efc54b04a18f9a113a65e3c9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.246217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4662ac1efc54b04a18f9a113a65e3c9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.248584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:31.267535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4662ac1efc54b04a18f9a113a65e3c9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.268563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4805a5df-cb7a-4ea7-9fc5-7ab171e68fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4662ac1efc54b04a18f9a113a65e3c9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.289783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1044/2106] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:30 2022] DELETE /identity/v3/users/d4662ac1efc54b04a18f9a113a65e3c9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:31.615918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.617578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.640131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a6f9e-9b23-4372-a788-315054e1b27b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.708814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:31.718846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.719621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.719910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.720191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] PATH_INFO: `/v3/projects/372fb575bdaa49d8b42288286108c7bb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.722292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=372fb575bdaa49d8b42288286108c7bb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.723699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.781336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372fb575bdaa49d8b42288286108c7bb', 'request_id': 'req-42cd653c-ff12-4e24-8afe-7ffb4595e84f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.788729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372fb575bdaa49d8b42288286108c7bb', 'request_id': 'req-42cd653c-ff12-4e24-8afe-7ffb4595e84f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.801327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372fb575bdaa49d8b42288286108c7bb', 'request_id': 'req-42cd653c-ff12-4e24-8afe-7ffb4595e84f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.818009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] The token cache is being invalidate because project 372fb575bdaa49d8b42288286108c7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:31.818756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.820108 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-42cd653c-ff12-4e24-8afe-7ffb4595e84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.825378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1063/2107] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:31 2022] DELETE /identity/v3/projects/372fb575bdaa49d8b42288286108c7bb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:31.831123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.831635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.831994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.856325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:31.858146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.942050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1045/2108] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:31.950663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.950811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.951092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.974186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:31.975743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.986408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1064/2109] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:31.993244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.007396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a941d3-d57a-4664-bde8-bda64fafe587 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.012239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.012854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.013136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.013410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.014315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:32.014706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.016990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.033025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1046/2110] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:32.038111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.048319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e75187-ce18-4a5e-826d-327509387349 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.053217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.053943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.054296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.055276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.056059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.058037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.087223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1065/2111] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:32.091488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.103857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d071a-485b-44ac-874d-f3f73ee91207 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.177965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.177965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.178296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.178697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.179449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.181543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.236797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1047/2112] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:32.237965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.249858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99cbf-16d6-4054-8ec2-339a849a74e9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.255773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.257135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.257640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.258109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.259163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.261136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.275500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1066/2113] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:32.278940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.291086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891d84c-f973-4761-9fb5-de4fc7f76554 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.296563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754a09-bb08-4f67-90d5-a6a1404d6540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.297191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754a09-bb08-4f67-90d5-a6a1404d6540 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.297625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754a09-bb08-4f67-90d5-a6a1404d6540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.298205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754a09-bb08-4f67-90d5-a6a1404d6540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.299042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9754a09-bb08-4f67-90d5-a6a1404d6540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.301364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9754a09-bb08-4f67-90d5-a6a1404d6540 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.338379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1048/2114] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:32.342619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.362120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2879ca-ca2e-4fc1-8244-313bc47aef33 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.370394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f63ef2-c4e7-4c2f-875b-ed717f790c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.371400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f63ef2-c4e7-4c2f-875b-ed717f790c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.377456 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f63ef2-c4e7-4c2f-875b-ed717f790c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.377794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f63ef2-c4e7-4c2f-875b-ed717f790c15 admin admin] PATH_INFO: `/v3/projects/6fb3652c48734333ab1fac5b9e2bf01b/users/e39de6275a644ee5917d422ca128a902/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.403320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f63ef2-c4e7-4c2f-875b-ed717f790c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb3652c48734333ab1fac5b9e2bf01b, user_id=e39de6275a644ee5917d422ca128a902, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.406514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f63ef2-c4e7-4c2f-875b-ed717f790c15 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.429298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1067/2115] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:32 2022] PUT /identity/v3/projects/6fb3652c48734333ab1fac5b9e2bf01b/users/e39de6275a644ee5917d422ca128a902/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:32.439848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f1b72-559f-4032-b653-8ea8052b8cc2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.440140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f1b72-559f-4032-b653-8ea8052b8cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.440418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f1b72-559f-4032-b653-8ea8052b8cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.486986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2b1f1b72-559f-4032-b653-8ea8052b8cc2 None None] MFA Rules not processed for user `e39de6275a644ee5917d422ca128a902`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:32.489121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2b1f1b72-559f-4032-b653-8ea8052b8cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:32.562980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:32.616118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1049/2116] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:34.662756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b801e0a-2d96-4e21-870b-94cdce21a3f1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.663341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b801e0a-2d96-4e21-870b-94cdce21a3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.663341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b801e0a-2d96-4e21-870b-94cdce21a3f1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.664896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1068/2117] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:21:34.693618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.694085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.695839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.730067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:34.733611 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.749362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b801e0a-2d96-4e21-870b-94cdce21a3f1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:34.751862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7b801e0a-2d96-4e21-870b-94cdce21a3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.868203 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1050/2118] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:34.870491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b801e0a-2d96-4e21-870b-94cdce21a3f1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:34.876305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:34.877174 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.877795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.878692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] PATH_INFO: `/v3/users/67fcd16d046c4df2a5dc99b43262375a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.880159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=67fcd16d046c4df2a5dc99b43262375a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:34.882847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:34.883530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:34.886190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.903787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee154652-98b4-4c59-9c19-956715d7da8d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:34.910976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:34.914874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.915221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.915535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.926289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:34.928128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:34.929871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:34.946521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1051/2119] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:34.966380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67fcd16d046c4df2a5dc99b43262375a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.967435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67fcd16d046c4df2a5dc99b43262375a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.984568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67fcd16d046c4df2a5dc99b43262375a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.994061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67fcd16d046c4df2a5dc99b43262375a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.002705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01293c18-22b3-4d39-8a95-e31a234b6fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67fcd16d046c4df2a5dc99b43262375a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.009752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1069/2120] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:34 2022] DELETE /identity/v3/users/67fcd16d046c4df2a5dc99b43262375a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:35.242526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.262697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b21b6-a242-4dc6-ba99-158e2ec17732 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.344756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.345406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.345720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.346006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] PATH_INFO: `/v3/projects/b9f5632d6b304db99e66877b0c329714` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.347987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f5632d6b304db99e66877b0c329714)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.349928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.407863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f5632d6b304db99e66877b0c329714', 'request_id': 'req-033e1e74-0fbc-4508-ba2f-a01bac21fc03'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.414079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f5632d6b304db99e66877b0c329714', 'request_id': 'req-033e1e74-0fbc-4508-ba2f-a01bac21fc03'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.417934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:35.421910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f5632d6b304db99e66877b0c329714', 'request_id': 'req-033e1e74-0fbc-4508-ba2f-a01bac21fc03'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.435704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] The token cache is being invalidate because project b9f5632d6b304db99e66877b0c329714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:35.435996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.436640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-033e1e74-0fbc-4508-ba2f-a01bac21fc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.438164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1052/2121] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:35 2022] DELETE /identity/v3/projects/b9f5632d6b304db99e66877b0c329714 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:35.443596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.443953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.444233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.463703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:35.464897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:35.530245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1070/2122] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:35.539616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.540006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.540366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.562111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:35.563266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:35.571252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1053/2123] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:35.575795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.586783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a891bd-7627-4d65-b579-0f4bbc6728ea None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.591434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.592207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.592416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.592610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.594695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:35.595006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.597458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.614183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1071/2124] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:35.618758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.628389 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843c78e-a59a-4023-aa0f-707ce7b56a73 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.633149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.633760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.634045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.634360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.634992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.636943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.665542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1054/2125] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:35.670227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.682501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7e40e-f972-49c6-bd56-233f317ca681 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.741048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.741681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.742020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.742298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.742989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.745007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.767112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1072/2126] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:35.771941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.783085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299759dc-c3f4-46b7-9971-05840c90709d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.788692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.789330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.789790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.790077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.790867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.792579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.804938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1055/2127] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:35.809399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.820559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4a8d-785b-4de6-8c72-ea891c32a8ae admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.825239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76867385-fe18-408d-90ff-9cc4d436280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.825852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76867385-fe18-408d-90ff-9cc4d436280c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.826129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76867385-fe18-408d-90ff-9cc4d436280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.826396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76867385-fe18-408d-90ff-9cc4d436280c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.827113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76867385-fe18-408d-90ff-9cc4d436280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.828737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76867385-fe18-408d-90ff-9cc4d436280c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.839845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1073/2128] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:35.844119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.856363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a746-6a8b-4193-9e38-c65a17a30117 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.861219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.861897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.862184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.862466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] PATH_INFO: `/v3/projects/7b6f2af7de6b4e888e702c78f655448a/users/c487e443eb274090bce842e855b015ec/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.879204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6f2af7de6b4e888e702c78f655448a, user_id=c487e443eb274090bce842e855b015ec, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.882130 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.895896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1056/2129] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:35 2022] PUT /identity/v3/projects/7b6f2af7de6b4e888e702c78f655448a/users/c487e443eb274090bce842e855b015ec/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:35.900955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.901242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.901547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.916952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:35.923267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:35.925156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:35.991932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1074/2130] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:36.002655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.006173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.077426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19a57-477e-4edc-afb5-284fdf120606 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.081187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad3a9-ea57-421a-b6d8-50f898e51bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.082336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad3a9-ea57-421a-b6d8-50f898e51bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.082623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad3a9-ea57-421a-b6d8-50f898e51bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.082901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad3a9-ea57-421a-b6d8-50f898e51bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.093058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad3a9-ea57-421a-b6d8-50f898e51bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.094548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad3a9-ea57-421a-b6d8-50f898e51bc0 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.105261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1057/2131] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:36.404818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.406278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.421578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd619ef4-044c-4d77-9b7f-e47ff69acd80 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.427198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.427966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.428319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.428659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] PATH_INFO: `/v3/users/655f0ce7fedb4436ba693052ce78623b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.429926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655f0ce7fedb4436ba693052ce78623b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.431990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.499518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655f0ce7fedb4436ba693052ce78623b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.500803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655f0ce7fedb4436ba693052ce78623b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.518899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655f0ce7fedb4436ba693052ce78623b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.527909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655f0ce7fedb4436ba693052ce78623b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.538068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655f0ce7fedb4436ba693052ce78623b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.547261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1075/2132] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:36 2022] DELETE /identity/v3/users/655f0ce7fedb4436ba693052ce78623b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:36.548974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.549258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.549533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.570690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:36.571826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.679038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1058/2133] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:36.690764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.692646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.776628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d39333-6e13-41d6-8243-c823647e3208 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.780690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.781461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.781741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.782017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.792442 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.794164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.807139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1076/2134] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:37.579600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:37.583177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.597614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385bba0-8f08-4f3a-8054-469a49893186 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:37.606232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:37.607075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.610989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.611406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] PATH_INFO: `/v3/projects/f4854754e7034ba38d2a27c4cb9b1c3e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.612917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4854754e7034ba38d2a27c4cb9b1c3e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:37.614910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.656741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:37.697709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4854754e7034ba38d2a27c4cb9b1c3e', 'request_id': 'req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.705122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4854754e7034ba38d2a27c4cb9b1c3e', 'request_id': 'req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.714788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4854754e7034ba38d2a27c4cb9b1c3e', 'request_id': 'req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.734480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] The token cache is being invalidate because project f4854754e7034ba38d2a27c4cb9b1c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:37.734480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.735138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.736929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1059/2135] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:37 2022] DELETE /identity/v3/projects/f4854754e7034ba38d2a27c4cb9b1c3e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:37.741115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:37.751156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7751f-e7bc-4ea0-8e98-ed6e9bff37de service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:37.781715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:37.828338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:37.829939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.829939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.829939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] PATH_INFO: `/v3/users/b17506bcb3e04b21a4b1cf4a1c0bbe41` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.830975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17506bcb3e04b21a4b1cf4a1c0bbe41)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:37.833642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.901300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17506bcb3e04b21a4b1cf4a1c0bbe41'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.902411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17506bcb3e04b21a4b1cf4a1c0bbe41'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.918023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17506bcb3e04b21a4b1cf4a1c0bbe41'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.927853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17506bcb3e04b21a4b1cf4a1c0bbe41'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.936433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f65df7eb-a774-4e85-a13b-63a3fd1e51e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17506bcb3e04b21a4b1cf4a1c0bbe41'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.950973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1077/2136] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:37 2022] DELETE /identity/v3/users/b17506bcb3e04b21a4b1cf4a1c0bbe41 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:38.414858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.416496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.448191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51fcbd-0d7e-4aa2-a09f-489af3f8b1a3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.556942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.557866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.558230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.558516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] PATH_INFO: `/v3/projects/189f3d72190e4f4ca8fc43e6d587c125` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.561514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189f3d72190e4f4ca8fc43e6d587c125)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.564092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.649547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189f3d72190e4f4ca8fc43e6d587c125', 'request_id': 'req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.670868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189f3d72190e4f4ca8fc43e6d587c125', 'request_id': 'req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.684647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189f3d72190e4f4ca8fc43e6d587c125', 'request_id': 'req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.718237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] The token cache is being invalidate because project 189f3d72190e4f4ca8fc43e6d587c125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:38.719355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.720567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-da4e34ec-1dbf-45ba-bc3c-9f4ac3208be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.727037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1060/2137] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:38 2022] DELETE /identity/v3/projects/189f3d72190e4f4ca8fc43e6d587c125 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:38.729406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.729772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.730126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.762580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:38.766543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.862131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1078/2138] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:38.871717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.872019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.872302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.897948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:38.907825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.918779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1061/2139] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:38.924709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.936729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2f03f-3a0e-435b-8239-0f48128aed23 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.948272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.949224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.949664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.950120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.951017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:38.951321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.953222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.975360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1079/2140] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:38.979836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.992102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d9222-748a-4041-81e8-5390ddf72ce0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.999160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.999590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.000016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.000329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.001162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.006113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.043834 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1062/2141] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:38 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:39.048271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.061130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b7417-edbf-46d6-b551-04109c04d024 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.113771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.180280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.181125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.181500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.181847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.182615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.185266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.239900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1080/2142] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:39.241351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.265848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98dda3-8c14-406a-a596-3aa7bc2153ab admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.278386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.279114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.279403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.279753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.280702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.282413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.305984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1063/2143] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:39.307835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.324941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcead30-e422-455d-a1e1-98bc9dfe4191 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.332951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19a59a-4377-4c77-907f-4dc6c24e689a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.338394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19a59a-4377-4c77-907f-4dc6c24e689a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.338794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19a59a-4377-4c77-907f-4dc6c24e689a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.339063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19a59a-4377-4c77-907f-4dc6c24e689a admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.339997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19a59a-4377-4c77-907f-4dc6c24e689a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.341811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19a59a-4377-4c77-907f-4dc6c24e689a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.368119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.368558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1081/2144] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:39.380381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.385094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249b8cc-6926-4ce3-a8be-4c5af6c8a3bc admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.396003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.396720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.397009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.397280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] PATH_INFO: `/v3/projects/f5d747b42b8c4d3eb3099561f0931cb4/users/7aebbe15b074456ca64c99d90b1bf41a/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.424500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d747b42b8c4d3eb3099561f0931cb4, user_id=7aebbe15b074456ca64c99d90b1bf41a, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.430440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.458637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1064/2145] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:39 2022] PUT /identity/v3/projects/f5d747b42b8c4d3eb3099561f0931cb4/users/7aebbe15b074456ca64c99d90b1bf41a/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:39.479190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.479506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.480616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.567861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] MFA Rules not processed for user `7aebbe15b074456ca64c99d90b1bf41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:39.569644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.698227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1082/2146] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:39.703339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.825972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a0941-1dff-4aba-9fb2-acf32f2d7c1d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.833177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.833826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.834114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.834392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.835674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.837987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.878629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1065/2147] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:39.885499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.900359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24eb7-315b-4443-9703-8de8c08b07d2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.970667 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.972223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.972392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.972907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.973458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.975628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.001339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1083/2148] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:40.005339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.018592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b03f8-17af-46fd-866b-8341aae2b9ad admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.023783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.024495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.024855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.025233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.026095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.027808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.039589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1066/2149] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:40.044840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.065815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb834c0-7a1c-4c87-ac61-4d8513d03671 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.070492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.071186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.071512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.071802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] PATH_INFO: `/v3/projects/5bf889c113bc46d78de539c65594716c/users/45d791a8164a4e689352be2238ec5e53/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.091151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf889c113bc46d78de539c65594716c, user_id=45d791a8164a4e689352be2238ec5e53, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.094242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.121554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1084/2150] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:40 2022] PUT /identity/v3/projects/5bf889c113bc46d78de539c65594716c/users/45d791a8164a4e689352be2238ec5e53/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:40.125633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.136957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffed264-d850-4f5c-9d45-063ba159fb65 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.213142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7974b0-2f04-4a85-ab8b-4ed07d3d42bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.213762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7974b0-2f04-4a85-ab8b-4ed07d3d42bf admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.214045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7974b0-2f04-4a85-ab8b-4ed07d3d42bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.214316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7974b0-2f04-4a85-ab8b-4ed07d3d42bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.215013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7974b0-2f04-4a85-ab8b-4ed07d3d42bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.216666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7974b0-2f04-4a85-ab8b-4ed07d3d42bf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.229173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1067/2151] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:40.238059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.257973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:40.273013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd733995-822d-47fb-87a2-f34abe77a6bf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.279126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.280581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.280899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.282507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] PATH_INFO: `/v3/projects/5bf889c113bc46d78de539c65594716c/users/45d791a8164a4e689352be2238ec5e53/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.287779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf889c113bc46d78de539c65594716c, user_id=45d791a8164a4e689352be2238ec5e53, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.303974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.319590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1085/2152] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:40 2022] PUT /identity/v3/projects/5bf889c113bc46d78de539c65594716c/users/45d791a8164a4e689352be2238ec5e53/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:40.333626 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b1aa7-ae44-4fbc-836d-58bb73600da0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.334648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b1aa7-ae44-4fbc-836d-58bb73600da0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.334648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b1aa7-ae44-4fbc-836d-58bb73600da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.368323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fa7b1aa7-ae44-4fbc-836d-58bb73600da0 None None] MFA Rules not processed for user `45d791a8164a4e689352be2238ec5e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:40.370122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fa7b1aa7-ae44-4fbc-836d-58bb73600da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.439869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:40.501220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1068/2153] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:40.513366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.515887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.629779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc4788-12a1-45b2-b528-79aeb56df510 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.634493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.635351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.635761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.636130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.648770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.651186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.666742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1086/2154] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:41.494620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fadf61-d68a-43f7-9a4d-fb6cbdbf2091 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:41.495534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fadf61-d68a-43f7-9a4d-fb6cbdbf2091 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:41.495975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fadf61-d68a-43f7-9a4d-fb6cbdbf2091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:41.566835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-31fadf61-d68a-43f7-9a4d-fb6cbdbf2091 None None] MFA Rules not processed for user `c487e443eb274090bce842e855b015ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:41.568158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-31fadf61-d68a-43f7-9a4d-fb6cbdbf2091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:41.616224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:41.666578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1069/2155] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:41.682070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:41.685354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:41.685685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:41.699305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b396b24-b0fa-411c-a629-73f6867e666c service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:41.704152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:41.704845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:41.705341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:41.705631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] PATH_INFO: `/v3/users/7aebbe15b074456ca64c99d90b1bf41a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:41.707418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aebbe15b074456ca64c99d90b1bf41a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:41.709422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:41.712359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] MFA Rules not processed for user `c487e443eb274090bce842e855b015ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:41.713620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:41.723533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1070/2156] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:41.752543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:41.754328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:41.779390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aebbe15b074456ca64c99d90b1bf41a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:41.780413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aebbe15b074456ca64c99d90b1bf41a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:41.804323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aebbe15b074456ca64c99d90b1bf41a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:41.814325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aebbe15b074456ca64c99d90b1bf41a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:41.824614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a43570e0-b26a-46fb-89c0-7e82a6a5b8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aebbe15b074456ca64c99d90b1bf41a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:41.839563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1087/2157] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:41 2022] DELETE /identity/v3/users/7aebbe15b074456ca64c99d90b1bf41a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:41.842050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:41.842412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:41.842790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:41.864337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:41.865646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:41.871077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:41.878817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14582ad4-fad4-4587-b061-28b4f0c7780e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:41.949659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:41.954218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1088/2158] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:41.964291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:41.965821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:41.974728 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:41.975372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:41.975699 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:41.975987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:41.977789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:42.062482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3ded-8ee1-4fbc-b5c1-516b609fff9a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:42.067371 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:42.068094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:42.068382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:42.068660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:42.080673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:42.081346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:42.083150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:42.087792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:42.104608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1089/2159] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:42.104928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1071/2160] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:42.473443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:42.475000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:42.489659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b05609-3b62-46cd-a11a-439b2c390260 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:42.494582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:42.495342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:42.495719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:42.496065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] PATH_INFO: `/v3/projects/f5d747b42b8c4d3eb3099561f0931cb4` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:42.497349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d747b42b8c4d3eb3099561f0931cb4)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:42.504189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:42.580576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d747b42b8c4d3eb3099561f0931cb4', 'request_id': 'req-f9c64f00-bc14-480a-8cc7-c150975aab62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.592949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d747b42b8c4d3eb3099561f0931cb4', 'request_id': 'req-f9c64f00-bc14-480a-8cc7-c150975aab62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.605245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d747b42b8c4d3eb3099561f0931cb4', 'request_id': 'req-f9c64f00-bc14-480a-8cc7-c150975aab62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.621797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] The token cache is being invalidate because project f5d747b42b8c4d3eb3099561f0931cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:42.622454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.622743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.624355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1090/2161] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:42 2022] DELETE /identity/v3/projects/f5d747b42b8c4d3eb3099561f0931cb4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:42.632613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:42.648138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757fd53-f26c-4ae4-b2ad-0c76e538d8bc service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:42.741659 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:42.742320 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:42.742604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:42.742886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] PATH_INFO: `/v3/users/45d791a8164a4e689352be2238ec5e53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:42.744136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45d791a8164a4e689352be2238ec5e53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:42.746249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:42.805313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45d791a8164a4e689352be2238ec5e53'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.806764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45d791a8164a4e689352be2238ec5e53'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.825800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45d791a8164a4e689352be2238ec5e53'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.835998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45d791a8164a4e689352be2238ec5e53'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.836736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0f888b22-8bbd-4cf0-907a-6265a0525fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45d791a8164a4e689352be2238ec5e53'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:42.864437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1072/2162] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:42 2022] DELETE /identity/v3/users/45d791a8164a4e689352be2238ec5e53 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:43.173150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.175186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.189156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c64f00-bc14-480a-8cc7-c150975aab62 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.234817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:43.271239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.271973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.272257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.272527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] PATH_INFO: `/v3/projects/5bf889c113bc46d78de539c65594716c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.273719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf889c113bc46d78de539c65594716c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.275682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.327331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf889c113bc46d78de539c65594716c', 'request_id': 'req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.333461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf889c113bc46d78de539c65594716c', 'request_id': 'req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.342124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf889c113bc46d78de539c65594716c', 'request_id': 'req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.361098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] The token cache is being invalidate because project 5bf889c113bc46d78de539c65594716c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:43.361498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.363477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7cf45dd-0137-4f24-84c5-984d7f2acf8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.365709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1091/2163] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:43 2022] DELETE /identity/v3/projects/5bf889c113bc46d78de539c65594716c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:43.370630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.372089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.372479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.399335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:43.400944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.411333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:43.494978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1073/2164] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:43.505779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.506595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.511255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.536543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:43.537798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.546055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1092/2165] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:43.551680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.561910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28e473-a800-452f-a245-1fe8e8301324 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.568119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.568748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.569040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.569335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.570042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:43.570350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.572498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.590822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1074/2166] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:43.592039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.602964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0e85-6374-4bd7-9166-ffae498a0435 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.607990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.608734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.609119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.609528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.610314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.613323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.647582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1093/2167] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:43 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:43.648921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.661174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fea3b3-4226-4de8-aa4d-33e01004b865 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.726848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.727511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.727796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.728094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.728799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.730818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.754403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1075/2168] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:43 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:43.758943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.770508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edfdb2-553a-4993-9188-0d26a00b9d94 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.773223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.775046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.776606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.777310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.777656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.778267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.778805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.780822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.797358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1094/2169] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:43.801962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.814255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1413f-5773-4cca-883a-ef8d792ad69f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.822403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.823150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.823722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.823833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.824588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.826152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.839810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1095/2170] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:43.843810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.855866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a80c4-f3b8-42e5-b8eb-2886b8de509c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.860285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7933ad-30b2-4bc5-984a-10e464740c49 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.862940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032fa89-6796-4bad-bf84-d75518296461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.863776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032fa89-6796-4bad-bf84-d75518296461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.864138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032fa89-6796-4bad-bf84-d75518296461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.864494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032fa89-6796-4bad-bf84-d75518296461 admin admin] PATH_INFO: `/v3/projects/c06397e1d65047a2a7187acdf6b61c59/users/042505f36f304d30ba40a9e49f6e7ede/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.864925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.865616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.865899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.866177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.868060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.888349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032fa89-6796-4bad-bf84-d75518296461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06397e1d65047a2a7187acdf6b61c59, user_id=042505f36f304d30ba40a9e49f6e7ede, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.894559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032fa89-6796-4bad-bf84-d75518296461 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.913537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1096/2171] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:43 2022] PUT /identity/v3/projects/c06397e1d65047a2a7187acdf6b61c59/users/042505f36f304d30ba40a9e49f6e7ede/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:43.927036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.928208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.928819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.933180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:43.958341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.959974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.964248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] MFA Rules not processed for user `042505f36f304d30ba40a9e49f6e7ede`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:43.965646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.978427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1076/2172] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:44.033059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:44.069819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1097/2173] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:44.074886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.091760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77988d3e-0b7c-4d96-a5fc-19d7081a7183 service glance] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.193553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.194415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.194769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.195055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.198552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.200826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.255593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1077/2174] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:44.257140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.283391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccd91b-d55c-46aa-a009-c8cbda1adaec None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.377411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.378295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.379221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.379221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.380252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.382475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.412452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1098/2175] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:44.416144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.429438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84630f6-2a7c-4fa1-b65b-f3a827035775 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.434661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.435561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.435980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.436327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.437127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.438849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.453707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1078/2176] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:44.458051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.469281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be707040-209b-4870-9c32-dbb4c9815d36 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.483009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.485066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.486844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.487215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] PATH_INFO: `/v3/projects/09d6677c54b246a2b35e0062fa4caf61/users/5205aea9370e485f971bb49087d73482/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.516841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d6677c54b246a2b35e0062fa4caf61, user_id=5205aea9370e485f971bb49087d73482, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.520198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.550493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1099/2177] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:44 2022] PUT /identity/v3/projects/09d6677c54b246a2b35e0062fa4caf61/users/5205aea9370e485f971bb49087d73482/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:44.555968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.576054 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.578273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:44.586292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b507ca-eff8-4471-b45a-6988078b15d3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.735421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65580df2-4eb4-4a1d-b261-722aeb1f1ae0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.748759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.748759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.748759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.748759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] PATH_INFO: `/v3/users/11f981b6b49f492ebc8e99791434a6b0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.749782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f981b6b49f492ebc8e99791434a6b0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.749782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.751887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.753070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.753662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.754292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.755265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.757328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.795345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1079/2178] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:44.796441 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.811439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad36d8-2890-4092-9e78-1438ef34f68f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.830364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f981b6b49f492ebc8e99791434a6b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.831792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f981b6b49f492ebc8e99791434a6b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.834922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e159aa-a3e4-4f17-9e89-3a1733fe3e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.842853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e159aa-a3e4-4f17-9e89-3a1733fe3e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.843923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e159aa-a3e4-4f17-9e89-3a1733fe3e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.844909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e159aa-a3e4-4f17-9e89-3a1733fe3e20 admin admin] PATH_INFO: `/v3/projects/09d6677c54b246a2b35e0062fa4caf61/users/5205aea9370e485f971bb49087d73482/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.845135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:44.853023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e159aa-a3e4-4f17-9e89-3a1733fe3e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d6677c54b246a2b35e0062fa4caf61, user_id=5205aea9370e485f971bb49087d73482, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.861352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e159aa-a3e4-4f17-9e89-3a1733fe3e20 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.882373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f981b6b49f492ebc8e99791434a6b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.895967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f981b6b49f492ebc8e99791434a6b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.906796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1080/2179] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:44 2022] PUT /identity/v3/projects/09d6677c54b246a2b35e0062fa4caf61/users/5205aea9370e485f971bb49087d73482/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:44.910618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-078b23d6-cecc-4c6f-ae25-b7e2dd41c0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f981b6b49f492ebc8e99791434a6b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.918250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee22a3-65b6-4bc3-ad53-eac366a89a84 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.918553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee22a3-65b6-4bc3-ad53-eac366a89a84 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.918901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee22a3-65b6-4bc3-ad53-eac366a89a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.921085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1100/2180] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:44 2022] DELETE /identity/v3/users/11f981b6b49f492ebc8e99791434a6b0 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:44.926139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.926415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.926687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.958505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4fee22a3-65b6-4bc3-ad53-eac366a89a84 None None] MFA Rules not processed for user `5205aea9370e485f971bb49087d73482`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:44.960060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4fee22a3-65b6-4bc3-ad53-eac366a89a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:44.987557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:44.991647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.007079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:45.100045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1081/2181] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:45.100045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.100045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.100045 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.144655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1101/2182] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:45.150378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] MFA Rules not processed for user `c487e443eb274090bce842e855b015ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:45.152094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.154716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:45.158395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.238823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1082/2183] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:45.239953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:45.255840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.258785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056c664-c8be-4d41-9363-d45ac04d8b2c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:45.266595 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.267284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.267713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.268023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.271006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.419195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600529-a4b6-4e44-b046-ae486747f645 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:45.424499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff19fd-1758-444f-a40a-f4f7841f2d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.425948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff19fd-1758-444f-a40a-f4f7841f2d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.426228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff19fd-1758-444f-a40a-f4f7841f2d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.426500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff19fd-1758-444f-a40a-f4f7841f2d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.435748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.437221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.454570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1102/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:45.457746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff19fd-1758-444f-a40a-f4f7841f2d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.463409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff19fd-1758-444f-a40a-f4f7841f2d7c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.493055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1083/2185] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:46.367114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.368608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:46.401009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d91f52-3dea-4193-b547-7df30973e5b9 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.408118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.408794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.409717 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.410000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] PATH_INFO: `/v3/projects/77a3c505bac3469a8eb78b36758dc1e7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.412207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a3c505bac3469a8eb78b36758dc1e7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.417018 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.442892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:46.500378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a3c505bac3469a8eb78b36758dc1e7', 'request_id': 'req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:46.514660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a3c505bac3469a8eb78b36758dc1e7', 'request_id': 'req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:46.524138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a3c505bac3469a8eb78b36758dc1e7', 'request_id': 'req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:46.549724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] The token cache is being invalidate because project 77a3c505bac3469a8eb78b36758dc1e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:46.550083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:46.553883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a6f657f1-aa4a-465e-a850-0b9cd09a88f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:46.555950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1103/2186] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:46 2022] DELETE /identity/v3/projects/77a3c505bac3469a8eb78b36758dc1e7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:46.561627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.562628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.562628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.605521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:46.607238 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:46.794594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1084/2187] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:46.804756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.805808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.807296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.901611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:46.908532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:46.947718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1104/2188] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:46.953879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.969094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3064fdb-3fc1-4075-a319-4d72d0c494a7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.979631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.980673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.981206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.982000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.983081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:46.984134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.987785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.023727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1085/2189] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:47.028871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.043205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37bd3-7154-4152-a19d-5307b0c8b36a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.049268 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.050549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.050835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.051315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.052059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.054396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.118511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1105/2190] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:47 2022] POST /identity/v3/projects => generated 363 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:47.124143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.130820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:47.140827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2043-3652-4954-b1d3-24a7258607ee admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.338495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.338953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.339236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.339566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.341741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.345831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.400779 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1086/2191] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:47 2022] POST /identity/v3/users => generated 282 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:47.401929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.419072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7679f5-a4cf-4d62-9f1a-df89bb672e7d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.424952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.425580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.425864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.426136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.426844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.428541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.443091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1106/2192] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:47.444396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.455580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d6add-4c50-415b-b54e-ef2a8917475d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.460801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635d047-6561-440d-9639-15b14769fdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.462471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d047-6561-440d-9639-15b14769fdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.462632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d047-6561-440d-9639-15b14769fdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.462820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d047-6561-440d-9639-15b14769fdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.463560 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d047-6561-440d-9639-15b14769fdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.465853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d047-6561-440d-9639-15b14769fdf0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.483500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1087/2193] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:47.485758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.498500 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9f09a-0e46-4b01-a6db-617b92427b58 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.506530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.507608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.507906 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.508183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] PATH_INFO: `/v3/projects/148a660f908b47cc91c4d8e53f7f5aa5/users/ac3098b6027d4308a5ef6e2a8d2c257b/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.525819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] RBAC: Authorizing `identity:create_grant(project_id=148a660f908b47cc91c4d8e53f7f5aa5, user_id=ac3098b6027d4308a5ef6e2a8d2c257b, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.528800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.544580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1107/2194] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:47 2022] PUT /identity/v3/projects/148a660f908b47cc91c4d8e53f7f5aa5/users/ac3098b6027d4308a5ef6e2a8d2c257b/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:47.555859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.556273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.556634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.585995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] MFA Rules not processed for user `ac3098b6027d4308a5ef6e2a8d2c257b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:47.587244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.667492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1088/2195] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:47.679325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.681211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.774092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa42492-5cfb-4efd-8145-a983aa18385b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.778585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b67488-99dc-45fd-93ae-28032c13c23f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.779217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b67488-99dc-45fd-93ae-28032c13c23f service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.779518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b67488-99dc-45fd-93ae-28032c13c23f service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.779798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b67488-99dc-45fd-93ae-28032c13c23f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.791270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b67488-99dc-45fd-93ae-28032c13c23f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.792681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b67488-99dc-45fd-93ae-28032c13c23f service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.808219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1108/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:48.463652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.465155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.542368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bce0e8-84c8-4af7-ac1c-22c0b70d6027 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.553209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.553993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.554285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.554573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] PATH_INFO: `/v3/users/042505f36f304d30ba40a9e49f6e7ede` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.557092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=042505f36f304d30ba40a9e49f6e7ede)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.559093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.575753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:48.615386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042505f36f304d30ba40a9e49f6e7ede'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.616346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042505f36f304d30ba40a9e49f6e7ede'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.631963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042505f36f304d30ba40a9e49f6e7ede'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.647579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042505f36f304d30ba40a9e49f6e7ede'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.648443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042505f36f304d30ba40a9e49f6e7ede'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.670350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1089/2197] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:48 2022] DELETE /identity/v3/users/042505f36f304d30ba40a9e49f6e7ede => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:48.675685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.676622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.676811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.708317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:48.709967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.793885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:48.821753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1109/2198] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:48.836278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.838866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.922678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18691f-478a-4ddd-9771-b3f53ef5cf3b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.927354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.928055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.928407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.928745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.938475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.940000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.951975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1090/2199] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:49.342316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.344836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:49.387238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edb70f-db3d-4f69-8cdd-7866260aef58 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.392324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.392993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.394273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.394558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] PATH_INFO: `/v3/projects/c06397e1d65047a2a7187acdf6b61c59` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.395957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06397e1d65047a2a7187acdf6b61c59)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.397908 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.469961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06397e1d65047a2a7187acdf6b61c59', 'request_id': 'req-85b09146-eb6c-4ad5-ae82-ded201863904'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.492256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06397e1d65047a2a7187acdf6b61c59', 'request_id': 'req-85b09146-eb6c-4ad5-ae82-ded201863904'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.505612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06397e1d65047a2a7187acdf6b61c59', 'request_id': 'req-85b09146-eb6c-4ad5-ae82-ded201863904'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.526727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] The token cache is being invalidate because project c06397e1d65047a2a7187acdf6b61c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:49.527033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.534701 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.537361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1110/2200] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:49 2022] DELETE /identity/v3/projects/c06397e1d65047a2a7187acdf6b61c59 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:49.542538 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.553706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db3e5c-2376-4873-a67e-563699e4789f service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.629023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.629843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.630390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.631018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] PATH_INFO: `/v3/users/5205aea9370e485f971bb49087d73482` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.633584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5205aea9370e485f971bb49087d73482)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.637312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.734293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:49.750304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5205aea9370e485f971bb49087d73482'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.751447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5205aea9370e485f971bb49087d73482'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.774925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5205aea9370e485f971bb49087d73482'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.790613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5205aea9370e485f971bb49087d73482'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.791651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1b98d376-b27b-468b-b19a-55b10aa27e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5205aea9370e485f971bb49087d73482'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:49.808408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1091/2201] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:49 2022] DELETE /identity/v3/users/5205aea9370e485f971bb49087d73482 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:50.364722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:50.367114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:50.399183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b09146-eb6c-4ad5-ae82-ded201863904 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:50.472900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:50.473474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:50.473759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:50.474222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] PATH_INFO: `/v3/projects/09d6677c54b246a2b35e0062fa4caf61` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:50.475619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d6677c54b246a2b35e0062fa4caf61)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:50.477574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:50.559670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d6677c54b246a2b35e0062fa4caf61', 'request_id': 'req-86397734-a405-4bd7-9296-3023c861e446'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:50.567766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d6677c54b246a2b35e0062fa4caf61', 'request_id': 'req-86397734-a405-4bd7-9296-3023c861e446'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:50.580524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d6677c54b246a2b35e0062fa4caf61', 'request_id': 'req-86397734-a405-4bd7-9296-3023c861e446'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:50.597573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:50.605008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] The token cache is being invalidate because project 09d6677c54b246a2b35e0062fa4caf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:50.605616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:50.616165 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-86397734-a405-4bd7-9296-3023c861e446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:50.618826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1111/2202] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:50 2022] DELETE /identity/v3/projects/09d6677c54b246a2b35e0062fa4caf61 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:50.624371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:50.624829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:50.625137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:50.670795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:50.672337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:50.750958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1092/2203] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:50.760086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:50.760384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:50.760620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:50.789736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:50.792542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:50.803314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1112/2204] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:50.808476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:50.821126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f56ad-ea2c-4374-ac47-334ec0e0c044 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:50.828512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:50.829556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:50.829843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:50.830121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:50.831427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:50.832839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:50.833858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:50.851651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1093/2205] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:50.855860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:50.878736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481e5d8-ab88-4c4a-aa92-cf6f6d02e54a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:50.884353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:50.884663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:50.884974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:50.886612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:50.887539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:50.889728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:50.931784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1113/2206] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:50 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:50.936559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:50.949318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e1353-6eb0-4da4-8486-365c904513f0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:51.062447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:51.067230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:51.067808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:51.068325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:51.069285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:51.072951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:51.117682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1094/2207] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:50 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:51.121176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:51.137125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513abe73-3d30-4840-abc1-3a837f5a7f0b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:51.143909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:51.144585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:51.144868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:51.145141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:51.146020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:51.147836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:51.171632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1114/2208] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:51.172055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:51.179241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23370-84c0-4de5-be89-9bb8bdd7ea23 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:51.188004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef4021-7118-406c-8b07-8af76b78f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:51.188745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4021-7118-406c-8b07-8af76b78f390 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:51.189256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4021-7118-406c-8b07-8af76b78f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:51.189708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4021-7118-406c-8b07-8af76b78f390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:51.191682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4021-7118-406c-8b07-8af76b78f390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:51.193248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4021-7118-406c-8b07-8af76b78f390 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:51.206663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1095/2209] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:51.211266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:51.223711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec6240-7b59-49f3-9f2e-ad7731898ec2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:51.233596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:51.234617 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:51.235077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:51.235587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] PATH_INFO: `/v3/projects/f275f615c2e9434da5b82c681faf52a9/users/c378759a40784380aa928e6b3cff7cfc/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:51.274618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f275f615c2e9434da5b82c681faf52a9, user_id=c378759a40784380aa928e6b3cff7cfc, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:51.279196 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:51.301308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1115/2210] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:51 2022] PUT /identity/v3/projects/f275f615c2e9434da5b82c681faf52a9/users/c378759a40784380aa928e6b3cff7cfc/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:51.317361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:51.317928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:51.318342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:51.359574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] MFA Rules not processed for user `c378759a40784380aa928e6b3cff7cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:51.361770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:51.371023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:51.475379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1096/2211] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:51.487395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:51.489557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:51.570184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210465-159f-48bd-9863-64a6f87535be admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:51.577773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b5fe7-9602-4333-b176-39d507c192f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:51.578545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5fe7-9602-4333-b176-39d507c192f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:51.578933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5fe7-9602-4333-b176-39d507c192f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:51.579271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5fe7-9602-4333-b176-39d507c192f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:51.589994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b5fe7-9602-4333-b176-39d507c192f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:51.592224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b5fe7-9602-4333-b176-39d507c192f7 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:51.608151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1116/2212] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:52.145278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:52.146892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:52.242946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432f329-bb0a-4300-a536-ef7d0b649852 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:52.249165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:52.249830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:52.250112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:52.250391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] PATH_INFO: `/v3/users/328847f6b46f483eb6ae0eaedcf5db6b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:52.253044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=328847f6b46f483eb6ae0eaedcf5db6b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:52.253740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:52.329955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '328847f6b46f483eb6ae0eaedcf5db6b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:52.331192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '328847f6b46f483eb6ae0eaedcf5db6b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:52.343500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '328847f6b46f483eb6ae0eaedcf5db6b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:52.352599 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '328847f6b46f483eb6ae0eaedcf5db6b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:52.353637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '328847f6b46f483eb6ae0eaedcf5db6b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:52.374069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1097/2213] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:52 2022] DELETE /identity/v3/users/328847f6b46f483eb6ae0eaedcf5db6b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:52.376183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:52.376631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:52.377361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:52.400447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:52.400789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:52.497836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1117/2214] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:52.504952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:52.507044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:52.553166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:52.589831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f74c03-3e08-4e98-9e5b-a130b11288b5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:52.593718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551291a-665b-455e-8cc5-85812000f776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:52.594392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551291a-665b-455e-8cc5-85812000f776 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:52.594674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551291a-665b-455e-8cc5-85812000f776 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:52.594947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551291a-665b-455e-8cc5-85812000f776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:52.605889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551291a-665b-455e-8cc5-85812000f776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:52.607277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551291a-665b-455e-8cc5-85812000f776 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:52.618910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1098/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:53.124713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.126232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:53.141779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aba463-c396-4d20-8c0e-d3db6918f10b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.148416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.149897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.150721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.151183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] PATH_INFO: `/v3/projects/db5014b6a5f7422ea900d0637585d96f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.152750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5014b6a5f7422ea900d0637585d96f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.154951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.215365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:53.244987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5014b6a5f7422ea900d0637585d96f', 'request_id': 'req-500db1d6-5a41-4754-8451-b067d2641a75'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:53.255290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5014b6a5f7422ea900d0637585d96f', 'request_id': 'req-500db1d6-5a41-4754-8451-b067d2641a75'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:53.263942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5014b6a5f7422ea900d0637585d96f', 'request_id': 'req-500db1d6-5a41-4754-8451-b067d2641a75'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:53.278851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] The token cache is being invalidate because project db5014b6a5f7422ea900d0637585d96f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:53.279157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:53.279948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-500db1d6-5a41-4754-8451-b067d2641a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:53.282089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1118/2216] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:53 2022] DELETE /identity/v3/projects/db5014b6a5f7422ea900d0637585d96f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:53.287996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.288564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.288564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.313495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:53.315008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:53.396887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1099/2217] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:53.406614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.407000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.407357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.429032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:53.430398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:53.441593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1119/2218] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:53.446009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.457845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876695f-b2c4-4cd4-a177-00c72ad17e47 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.462977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.463798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.464126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.464466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.465183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:53.465531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.467263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.488219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1100/2219] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:53.489838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.499864 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4016db4-1077-42c1-810b-744d8ab158b6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.505114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.505753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.506039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.506647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.507246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.509247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.547267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1120/2220] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:53 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:53.553466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.564190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f095a0-f9c2-4159-85a1-6afc97b7df6e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.650365 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.651167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.652244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.652689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.653855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.657684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.704179 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1101/2221] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:53 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:53.709292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.721958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd1bfb-6633-43b5-a7cf-500aa93f926c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.729015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.729873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.730509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.730936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.732511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.734470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.752808 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.754296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1121/2222] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:53.769288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed3d12-ea22-4bb1-b702-40d8ebdbe709 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.775124 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb125d65-5827-4b84-91ef-1d9700551d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.776225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb125d65-5827-4b84-91ef-1d9700551d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.776689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb125d65-5827-4b84-91ef-1d9700551d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.777195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb125d65-5827-4b84-91ef-1d9700551d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.778206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb125d65-5827-4b84-91ef-1d9700551d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.780693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb125d65-5827-4b84-91ef-1d9700551d49 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.797915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1102/2223] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:53.805290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:53.826351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f9ae0-a7ea-4d02-ac38-edb6ea3cdc3f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:53.837453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:53.838513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.838973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.839616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] PATH_INFO: `/v3/projects/696dc82ba30a4161be80a435d94c605d/users/efb088ac0d9841a2aacc90818ee972de/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.861923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696dc82ba30a4161be80a435d94c605d, user_id=efb088ac0d9841a2aacc90818ee972de, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:53.866326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:53.890527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1122/2224] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:53 2022] PUT /identity/v3/projects/696dc82ba30a4161be80a435d94c605d/users/efb088ac0d9841a2aacc90818ee972de/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:53.910180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:53.910548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:53.910892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:53.945405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] MFA Rules not processed for user `efb088ac0d9841a2aacc90818ee972de`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:53.946757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:54.005261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:54.082947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1103/2225] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:54.106907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:54.117073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:54.198789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb587a-d02f-48e1-a9b8-ea8174413214 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:54.208303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39d516-3725-4e6c-a8c5-88d1a4eb4db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:54.209270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39d516-3725-4e6c-a8c5-88d1a4eb4db1 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:54.209735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39d516-3725-4e6c-a8c5-88d1a4eb4db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:54.212072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39d516-3725-4e6c-a8c5-88d1a4eb4db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:54.229615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39d516-3725-4e6c-a8c5-88d1a4eb4db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:54.232195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39d516-3725-4e6c-a8c5-88d1a4eb4db1 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:54.255901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1123/2226] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:55.137462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:55.139514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:55.218127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b1f7-c504-4c66-a87f-7b620e5034ee None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:55.223243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:55.223856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:55.224198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:55.224479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] PATH_INFO: `/v3/users/c378759a40784380aa928e6b3cff7cfc` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:55.225800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c378759a40784380aa928e6b3cff7cfc)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:55.227927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:55.288042 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c378759a40784380aa928e6b3cff7cfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:55.288747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c378759a40784380aa928e6b3cff7cfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:55.307755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c378759a40784380aa928e6b3cff7cfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:55.318532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c378759a40784380aa928e6b3cff7cfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:55.319276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c378759a40784380aa928e6b3cff7cfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:55.344102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1104/2227] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:55 2022] DELETE /identity/v3/users/c378759a40784380aa928e6b3cff7cfc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:55.349741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:55.350643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:55.350820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:55.375120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:55.377314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:55.463217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:55.482901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1124/2228] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:55.507710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:55.509727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:55.617843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:55.631090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e3402-5b30-4cc5-b09a-3f1fa2beeb8c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:55.636623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb57ef1-c271-4cf6-a76a-06fde72402eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:55.637969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb57ef1-c271-4cf6-a76a-06fde72402eb service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:55.638531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb57ef1-c271-4cf6-a76a-06fde72402eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:55.638809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb57ef1-c271-4cf6-a76a-06fde72402eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:55.650459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb57ef1-c271-4cf6-a76a-06fde72402eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:55.651963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb57ef1-c271-4cf6-a76a-06fde72402eb service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:55.669957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1105/2229] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:56.122611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.124223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.141903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d355-8bad-4a35-a21a-d00af6dbc34e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.148587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.149884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.149884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.150238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] PATH_INFO: `/v3/projects/f275f615c2e9434da5b82c681faf52a9` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.153408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f275f615c2e9434da5b82c681faf52a9)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.156809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.236278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f275f615c2e9434da5b82c681faf52a9', 'request_id': 'req-6259f14f-eec7-44a3-9aaa-623181b357ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.248564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f275f615c2e9434da5b82c681faf52a9', 'request_id': 'req-6259f14f-eec7-44a3-9aaa-623181b357ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.263279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f275f615c2e9434da5b82c681faf52a9', 'request_id': 'req-6259f14f-eec7-44a3-9aaa-623181b357ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.279337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] The token cache is being invalidate because project f275f615c2e9434da5b82c681faf52a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:56.279676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.280467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6259f14f-eec7-44a3-9aaa-623181b357ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.283092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1125/2230] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:56 2022] DELETE /identity/v3/projects/f275f615c2e9434da5b82c681faf52a9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:56.289194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.289678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.289904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.316326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:56.317645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.406355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1106/2231] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:56.409913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.410441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.410633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.438608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:56.441231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.450768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1126/2232] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:56.455510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.473799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12218d3d-46ca-4042-a308-8149c780a5f3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.478907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.479583 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.479870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.480150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.480863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:56.481233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.482789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.497199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1107/2233] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:56.501574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.511419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d520a-0fe3-4a4f-a200-82e3fbffaeb1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.516618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.519173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.519173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.519484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.520266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.522185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.563133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1127/2234] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:56 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:56.567440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.580875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6000b8-d86e-477f-b59f-2ab53f6ec2a2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.653011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.653676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.653960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.654242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.654924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.657059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.688459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1108/2235] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:56 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:56.692680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.705095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80934d74-ca4f-4f61-beb8-f1db5395bc89 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.710394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.711029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.711313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.711621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.712329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.713958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.729353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1128/2236] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:56.734093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.746644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f69af-f90a-4867-a85c-3369d998d842 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.751975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d869c-0129-411d-a9cd-dfff50ceadfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.752769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d869c-0129-411d-a9cd-dfff50ceadfc admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.753055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d869c-0129-411d-a9cd-dfff50ceadfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.753331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d869c-0129-411d-a9cd-dfff50ceadfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.754067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d869c-0129-411d-a9cd-dfff50ceadfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.755768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d869c-0129-411d-a9cd-dfff50ceadfc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.768589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1109/2237] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:56.772851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.782850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbb6aa-70e3-4e55-851b-af56efcb6687 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.788334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.789036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.789319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.789593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] PATH_INFO: `/v3/projects/d6dd5eef6c62467ab15a23233e366185/users/bbbdf80e14b647cbbdb3d292d534d987/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.807641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dd5eef6c62467ab15a23233e366185, user_id=bbbdf80e14b647cbbdb3d292d534d987, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.810285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.823222 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1129/2238] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:56 2022] PUT /identity/v3/projects/d6dd5eef6c62467ab15a23233e366185/users/bbbdf80e14b647cbbdb3d292d534d987/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:56.828530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aafe4-c5d2-4cde-97b6-29d0cc3a74e0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.828958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aafe4-c5d2-4cde-97b6-29d0cc3a74e0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.829285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aafe4-c5d2-4cde-97b6-29d0cc3a74e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.849507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7f6aafe4-c5d2-4cde-97b6-29d0cc3a74e0 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:56.850731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7f6aafe4-c5d2-4cde-97b6-29d0cc3a74e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.899924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:56.914759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1110/2239] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:56.924487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.926212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.984762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21b99e-1677-4b7d-930a-147eafc63e8b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.988505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.989482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.989482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.989662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.999434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:57.000919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:57.012343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1130/2240] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:57.749462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba864f-4b1e-497b-bc7a-b81719fedb0c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:57.750109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba864f-4b1e-497b-bc7a-b81719fedb0c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:57.750468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba864f-4b1e-497b-bc7a-b81719fedb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:57.805496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-02ba864f-4b1e-497b-bc7a-b81719fedb0c None None] MFA Rules not processed for user `bbbdf80e14b647cbbdb3d292d534d987`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:57.807875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-02ba864f-4b1e-497b-bc7a-b81719fedb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:57.926315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1111/2241] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:21:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:57.935218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:57.938158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:57.982832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:58.027398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde21e7-7265-4a52-8d8d-c3455b02e78c service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:58.031935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:58.033070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:58.033070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:58.033259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:58.048243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:58.051088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:58.066812 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1131/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:03.728144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d8b3-3455-4462-8998-74abfcf761c2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:03.728708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d8b3-3455-4462-8998-74abfcf761c2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:03.728708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345d8b3-3455-4462-8998-74abfcf761c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:03.748586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e345d8b3-3455-4462-8998-74abfcf761c2 None None] MFA Rules not processed for user `bbbdf80e14b647cbbdb3d292d534d987`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:03.749851 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e345d8b3-3455-4462-8998-74abfcf761c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:03.757752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1112/2243] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:08.062279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=16 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:13.742813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=38 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:29.534831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:29.536539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:29.551714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952fa25-f5c4-4104-9c51-625570ca968a service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:29.557011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:29.557916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:29.558202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:29.558477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] PATH_INFO: `/v3/users/efb088ac0d9841a2aacc90818ee972de` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:29.559714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb088ac0d9841a2aacc90818ee972de)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:29.561691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:29.641515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb088ac0d9841a2aacc90818ee972de'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:29.655619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb088ac0d9841a2aacc90818ee972de'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:29.677244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb088ac0d9841a2aacc90818ee972de'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:29.688728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb088ac0d9841a2aacc90818ee972de'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:29.697523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb088ac0d9841a2aacc90818ee972de'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:29.706055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1132/2244] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:29 2022] DELETE /identity/v3/users/efb088ac0d9841a2aacc90818ee972de => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:29.713578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176643-48c4-4919-975a-49705d61d369 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:29.714065 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176643-48c4-4919-975a-49705d61d369 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:29.714171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176643-48c4-4919-975a-49705d61d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:29.734504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-32176643-48c4-4919-975a-49705d61d369 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:29.738627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-32176643-48c4-4919-975a-49705d61d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:29.867623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1113/2245] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:29.879305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:29.881147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:29.993298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d4325-bfc4-4762-acf1-3fc5fd7acbe9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:29.998173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70c4cc-9c1c-4ded-b2a5-373cf42aef74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:29.999072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70c4cc-9c1c-4ded-b2a5-373cf42aef74 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:29.999366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70c4cc-9c1c-4ded-b2a5-373cf42aef74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:29.999725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70c4cc-9c1c-4ded-b2a5-373cf42aef74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:30.024597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70c4cc-9c1c-4ded-b2a5-373cf42aef74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:30.024597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70c4cc-9c1c-4ded-b2a5-373cf42aef74 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:30.030159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1133/2246] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:30.358641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176643-48c4-4919-975a-49705d61d369 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:30.360288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-32176643-48c4-4919-975a-49705d61d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:30.378096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176643-48c4-4919-975a-49705d61d369 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:30.385948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:30.386903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:30.387342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:30.387668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] PATH_INFO: `/v3/projects/696dc82ba30a4161be80a435d94c605d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:30.389069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696dc82ba30a4161be80a435d94c605d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:30.391133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:30.446480 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696dc82ba30a4161be80a435d94c605d', 'request_id': 'req-7a7be9c4-9391-4c79-81e0-3894e47f0146'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:30.453361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696dc82ba30a4161be80a435d94c605d', 'request_id': 'req-7a7be9c4-9391-4c79-81e0-3894e47f0146'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:30.461153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696dc82ba30a4161be80a435d94c605d', 'request_id': 'req-7a7be9c4-9391-4c79-81e0-3894e47f0146'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:30.474215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] The token cache is being invalidate because project 696dc82ba30a4161be80a435d94c605d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:30.474671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:30.475518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7a7be9c4-9391-4c79-81e0-3894e47f0146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:30.477068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1114/2247] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:30 2022] DELETE /identity/v3/projects/696dc82ba30a4161be80a435d94c605d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:30.708504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:30.709121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:30.709121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:30.729082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] MFA Rules not processed for user `9eda4e19122444cf9cb2b6941adff9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:30.730288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:30.788760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1134/2248] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:35.691651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:35.692166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:35.692261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:35.718977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:35.720280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:35.819752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1115/2249] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:35.884158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:35.886674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:35.937462 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:36.021466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ebca-3d3f-4b76-94e7-4dd4161d5817 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:36.029795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bba52-24bc-49ba-ad0d-0de93cf5c7a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:36.030548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bba52-24bc-49ba-ad0d-0de93cf5c7a6 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:36.031599 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bba52-24bc-49ba-ad0d-0de93cf5c7a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:36.031910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bba52-24bc-49ba-ad0d-0de93cf5c7a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:36.046036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bba52-24bc-49ba-ad0d-0de93cf5c7a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:36.047690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bba52-24bc-49ba-ad0d-0de93cf5c7a6 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:36.057858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1135/2250] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:22:45.796676 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:46.054435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=23 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:50.343886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:50.347047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:50.444810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7b6af-cd62-42a6-8d9d-fde9260b65a3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:50.450718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:50.451698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:50.452160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:50.452637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] PATH_INFO: `/v3/users/e39de6275a644ee5917d422ca128a902` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:50.454652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39de6275a644ee5917d422ca128a902)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:50.456723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:50.537424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39de6275a644ee5917d422ca128a902'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:50.538234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39de6275a644ee5917d422ca128a902'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:50.561863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39de6275a644ee5917d422ca128a902'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:50.573917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39de6275a644ee5917d422ca128a902'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:50.575095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39de6275a644ee5917d422ca128a902'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:50.601280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1116/2251] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:50 2022] DELETE /identity/v3/users/e39de6275a644ee5917d422ca128a902 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:50.606695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:50.607880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:50.608265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:50.629914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:50.631192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:50.724307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1136/2252] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:50.734498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:50.736004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:50.836905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2abe94-4298-4f6e-b68e-d0cc28ac471a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:50.841662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e566c6b-948c-421e-ba2a-06a3bde49d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:50.842581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e566c6b-948c-421e-ba2a-06a3bde49d62 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:50.843029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e566c6b-948c-421e-ba2a-06a3bde49d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:50.843439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e566c6b-948c-421e-ba2a-06a3bde49d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:50.860202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e566c6b-948c-421e-ba2a-06a3bde49d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:50.861768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e566c6b-948c-421e-ba2a-06a3bde49d62 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:50.875771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1117/2253] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:22:51.299125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.304890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:51.331693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08b4ee-1d58-4f5f-8a6c-c6d341b0dd3b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.340566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.341961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.342188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.348758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] PATH_INFO: `/v3/projects/6fb3652c48734333ab1fac5b9e2bf01b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.352183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb3652c48734333ab1fac5b9e2bf01b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.354930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:51.420726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb3652c48734333ab1fac5b9e2bf01b', 'request_id': 'req-a295250e-8d46-4fbd-8287-958fba707099'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:51.427813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb3652c48734333ab1fac5b9e2bf01b', 'request_id': 'req-a295250e-8d46-4fbd-8287-958fba707099'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:51.439841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb3652c48734333ab1fac5b9e2bf01b', 'request_id': 'req-a295250e-8d46-4fbd-8287-958fba707099'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:51.458045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] The token cache is being invalidate because project 6fb3652c48734333ab1fac5b9e2bf01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:51.458342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:51.459747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a295250e-8d46-4fbd-8287-958fba707099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:51.466123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1137/2254] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:51 2022] DELETE /identity/v3/projects/6fb3652c48734333ab1fac5b9e2bf01b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:51.468576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.469488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.470043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.516368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:51.519113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:51.552887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:51.625906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1118/2255] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:51.628034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.629189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.629670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.655730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:51.658155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:51.674116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1138/2256] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:51.679217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.691007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d37275-164f-4a6b-9f2f-5e90dadf4036 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.696036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.696702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.696988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.697262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.698028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:51.698358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.700120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:51.714804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1119/2257] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:22:51.719853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.730689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd201b-8445-4d5f-a122-9c907417e8d9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.736154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.736801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.737094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.737369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.738076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.740026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:51.775728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1139/2258] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:51 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:51.777447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.789386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa577abf-ef49-4111-aace-12f4aff9d08c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.855598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.856244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.856709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.856998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.857686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.859760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:51.883446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1120/2259] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:51 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:51.887777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.899094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ff884-87dd-42f3-9ec2-cedc86650b71 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.904072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.904726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.904888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.905162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.905869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.907665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:51.918414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1140/2260] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:51.922449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.933271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a05f9-371d-48e7-8152-b472e5a6b5a8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.939730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadff7e2-8488-45cb-a060-60665edd6011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.940360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadff7e2-8488-45cb-a060-60665edd6011 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.940646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadff7e2-8488-45cb-a060-60665edd6011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.940923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadff7e2-8488-45cb-a060-60665edd6011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.941805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadff7e2-8488-45cb-a060-60665edd6011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.943658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadff7e2-8488-45cb-a060-60665edd6011 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:51.959590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1121/2261] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:51.960709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:51.971479 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475333e-f655-414e-9f7e-d131a16fc68f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:51.976866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:51.977576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:51.977860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:51.978133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] PATH_INFO: `/v3/projects/27f6ab4a05704f8c97a5e1883becb247/users/a5711f44f73e4925ae52a784f43a2a1c/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:51.995406 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f6ab4a05704f8c97a5e1883becb247, user_id=a5711f44f73e4925ae52a784f43a2a1c, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:51.997723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:52.027310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1141/2262] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:51 2022] PUT /identity/v3/projects/27f6ab4a05704f8c97a5e1883becb247/users/a5711f44f73e4925ae52a784f43a2a1c/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:52.039322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cd49d-c2c2-4e96-b2cf-0efc2ecda973 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:52.039663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cd49d-c2c2-4e96-b2cf-0efc2ecda973 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:52.040901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cd49d-c2c2-4e96-b2cf-0efc2ecda973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:52.076353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1a0cd49d-c2c2-4e96-b2cf-0efc2ecda973 None None] MFA Rules not processed for user `a5711f44f73e4925ae52a784f43a2a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:52.077947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a0cd49d-c2c2-4e96-b2cf-0efc2ecda973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:52.186691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1122/2263] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:52.188467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:52.191357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:52.222926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:52.299952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864127ee-82f7-4cc0-a418-881e5774caa5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:52.307956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7077a-6eff-47ca-8856-58322a230e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:52.308486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7077a-6eff-47ca-8856-58322a230e1b service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:52.309033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7077a-6eff-47ca-8856-58322a230e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:52.309357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7077a-6eff-47ca-8856-58322a230e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:52.319944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7077a-6eff-47ca-8856-58322a230e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:52.321600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7077a-6eff-47ca-8856-58322a230e1b service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:52.336555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1142/2264] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:22:54.863566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:54.864192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:54.864192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:54.888868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] MFA Rules not processed for user `a5711f44f73e4925ae52a784f43a2a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:54.890149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:54.899884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1123/2265] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:55.308568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4283f1-4797-4cfd-bde1-b552a990d234 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:55.309160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4283f1-4797-4cfd-bde1-b552a990d234 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:55.309258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4283f1-4797-4cfd-bde1-b552a990d234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:55.332232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4c4283f1-4797-4cfd-bde1-b552a990d234 None None] MFA Rules not processed for user `bbbdf80e14b647cbbdb3d292d534d987`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:55.333718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4c4283f1-4797-4cfd-bde1-b552a990d234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:55.399570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1143/2266] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:55.915184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:55.918649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:55.940479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:56.020353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd689a36-4f8f-42f1-b2c2-57fa451430c4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:56.025752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:56.026420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:56.026904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:56.027554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] PATH_INFO: `/v3/users/ac3098b6027d4308a5ef6e2a8d2c257b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:56.029693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3098b6027d4308a5ef6e2a8d2c257b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:56.032558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:56.174095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3098b6027d4308a5ef6e2a8d2c257b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:56.174788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3098b6027d4308a5ef6e2a8d2c257b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:56.195554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3098b6027d4308a5ef6e2a8d2c257b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:56.208743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3098b6027d4308a5ef6e2a8d2c257b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:56.209749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3098b6027d4308a5ef6e2a8d2c257b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:56.266868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1124/2267] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:55 2022] DELETE /identity/v3/users/ac3098b6027d4308a5ef6e2a8d2c257b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:56.272787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:56.273163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:56.273449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:56.312935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:56.314203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:56.474414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1144/2268] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:56.518740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:56.521467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:56.651168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552d1f-559f-4723-8753-7095b921d2b3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:56.657296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462416d3-dad2-4b5a-97e3-b460112962b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:56.658192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462416d3-dad2-4b5a-97e3-b460112962b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:56.658482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462416d3-dad2-4b5a-97e3-b460112962b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:56.658777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462416d3-dad2-4b5a-97e3-b460112962b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:56.678063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462416d3-dad2-4b5a-97e3-b460112962b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:56.679744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462416d3-dad2-4b5a-97e3-b460112962b0 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:56.694498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1125/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:57.028922 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.030384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:57.046387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6362d-5f95-4a4d-9681-5f3eec103e5f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.051310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.052003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.052372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.052653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] PATH_INFO: `/v3/projects/148a660f908b47cc91c4d8e53f7f5aa5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.053868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=148a660f908b47cc91c4d8e53f7f5aa5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.055975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.107067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148a660f908b47cc91c4d8e53f7f5aa5', 'request_id': 'req-cf3e227e-4eaf-40f4-bfee-5269bf346656'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:57.112803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148a660f908b47cc91c4d8e53f7f5aa5', 'request_id': 'req-cf3e227e-4eaf-40f4-bfee-5269bf346656'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:57.116611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:57.121097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148a660f908b47cc91c4d8e53f7f5aa5', 'request_id': 'req-cf3e227e-4eaf-40f4-bfee-5269bf346656'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:57.135169 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] The token cache is being invalidate because project 148a660f908b47cc91c4d8e53f7f5aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:57.135450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:57.136450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf3e227e-4eaf-40f4-bfee-5269bf346656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:57.137926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1145/2270] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:57 2022] DELETE /identity/v3/projects/148a660f908b47cc91c4d8e53f7f5aa5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:57.148174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.148717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.148717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.169547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:57.170783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:57.201204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:57.236120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1126/2271] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:57.244084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.244370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.244647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.265430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:57.267278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:57.275870 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1146/2272] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:57.282379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.293061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5898-319d-424a-a449-9ee647c00d83 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.298040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.298840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.299221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.300067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.301064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:57.301470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.304128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.319093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1127/2273] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:22:57.323091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.333491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5655b-e17e-4324-98eb-6964ed34950c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.338460 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.339147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.339430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.339730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.340531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.342388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.371062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1147/2274] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:57 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:57.375813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.385753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3863a9-1849-4518-b8ce-6c52e2c505cf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.443522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.444145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.444466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.444744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.445411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.447346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.470576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1128/2275] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:57 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:57.475019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.485980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976dfd-b70b-4abb-bde3-033bf19c4929 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.490532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.491138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.491419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.491719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.492436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.493992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.505298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1148/2276] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:57.510796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.522100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961db1c-841a-4987-b4bd-fd071fbb98a2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.527106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9085-70c5-4a60-9fdc-b3bbf2aab8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.527706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9085-70c5-4a60-9fdc-b3bbf2aab8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.527989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9085-70c5-4a60-9fdc-b3bbf2aab8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.528267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9085-70c5-4a60-9fdc-b3bbf2aab8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.528971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e9085-70c5-4a60-9fdc-b3bbf2aab8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.530567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e9085-70c5-4a60-9fdc-b3bbf2aab8ea admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.542395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1129/2277] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:57.546441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.557153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f6985-d0f8-4aca-b800-2fdc896db058 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.561881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.563110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.563393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.563710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] PATH_INFO: `/v3/projects/b3e0acab6925474ca4c9712f33465207/users/20906561720a4d5d8b099725da89c9f5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.578817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e0acab6925474ca4c9712f33465207, user_id=20906561720a4d5d8b099725da89c9f5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.581643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.595420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1149/2278] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:57 2022] PUT /identity/v3/projects/b3e0acab6925474ca4c9712f33465207/users/20906561720a4d5d8b099725da89c9f5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:57.605862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc795-3523-4713-96ea-4444b06de126 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.606162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc795-3523-4713-96ea-4444b06de126 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.606898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc795-3523-4713-96ea-4444b06de126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.639675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-743cc795-3523-4713-96ea-4444b06de126 None None] MFA Rules not processed for user `20906561720a4d5d8b099725da89c9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:57.641122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-743cc795-3523-4713-96ea-4444b06de126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:57.726194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1130/2279] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:57.738833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.741350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:57.808596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84ffaa-2a2d-4cd2-b738-7e39efc6e702 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.812346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759809b-e559-4c61-84b2-95c71df676d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.812962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759809b-e559-4c61-84b2-95c71df676d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.813239 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759809b-e559-4c61-84b2-95c71df676d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.813512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759809b-e559-4c61-84b2-95c71df676d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.821735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759809b-e559-4c61-84b2-95c71df676d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.823151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759809b-e559-4c61-84b2-95c71df676d1 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.832050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1150/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:22:57.856910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cc795-3523-4713-96ea-4444b06de126 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:57.887768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:57.921416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cc795-3523-4713-96ea-4444b06de126 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:57.926914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:57.927604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:57.927889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:57.928166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] PATH_INFO: `/v3/users/20906561720a4d5d8b099725da89c9f5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:57.929577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20906561720a4d5d8b099725da89c9f5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:57.931843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:57.991897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20906561720a4d5d8b099725da89c9f5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:57.992594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20906561720a4d5d8b099725da89c9f5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.008900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20906561720a4d5d8b099725da89c9f5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.016998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20906561720a4d5d8b099725da89c9f5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.018255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20906561720a4d5d8b099725da89c9f5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.034257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1131/2281] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:57 2022] DELETE /identity/v3/users/20906561720a4d5d8b099725da89c9f5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:58.038993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.039348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.039755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.060837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:58.062065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.135662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1151/2282] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:58.145824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.147312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.206536 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdf580-ee7b-48dd-b7c6-c25a8474d54e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.210387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812921c6-10a3-4707-948b-a640983d1f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.210999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812921c6-10a3-4707-948b-a640983d1f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.211278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812921c6-10a3-4707-948b-a640983d1f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.211587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812921c6-10a3-4707-948b-a640983d1f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.221950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812921c6-10a3-4707-948b-a640983d1f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.223872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812921c6-10a3-4707-948b-a640983d1f80 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.234546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1132/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:22:58.512655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.514240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.528201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bfbac-16bb-4873-9fdc-d76f700cc533 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.540997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.541698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.541976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.542279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] PATH_INFO: `/v3/projects/b3e0acab6925474ca4c9712f33465207` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.543512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e0acab6925474ca4c9712f33465207)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.547519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.581082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:58.599393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e0acab6925474ca4c9712f33465207', 'request_id': 'req-57c51476-d9e9-4366-9394-7569561b7ac8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.605692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e0acab6925474ca4c9712f33465207', 'request_id': 'req-57c51476-d9e9-4366-9394-7569561b7ac8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.613115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e0acab6925474ca4c9712f33465207', 'request_id': 'req-57c51476-d9e9-4366-9394-7569561b7ac8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.624049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] The token cache is being invalidate because project b3e0acab6925474ca4c9712f33465207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:58.624335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.625521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-57c51476-d9e9-4366-9394-7569561b7ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.627034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1152/2284] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:58 2022] DELETE /identity/v3/projects/b3e0acab6925474ca4c9712f33465207 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:58.632008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.632307 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.632589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.656246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:58.657496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.704589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:58.720990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1133/2285] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:58.729880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.730177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.730463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.752635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:58.754296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.763511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1153/2286] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:58.768387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.778557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50130943-db12-4e54-bcc6-b221d8df9fed None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.783322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.783978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.784261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.784534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.785272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:58.785578 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.787199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.800021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1134/2287] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:22:58.804464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.819798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ae06-f1ad-49d5-9486-832308b28379 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.823140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.823970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.824409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.824726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.825499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.827397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.858840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1154/2288] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:58 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:58.863067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.876358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4d0e2-cf98-418c-b379-6c58e35434d2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.953631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.954263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.954547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.954822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.955518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.957496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.981980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1135/2289] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:58 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:58.990112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:59.001011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d546b-bcc4-4a35-8978-b6c244d626cc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:59.005660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:59.006269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.006548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.006827 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.007566 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:59.009159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:59.019725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1155/2290] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:59.023709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:59.033928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80643-a4a5-4107-bbe5-2785f291ca1c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:59.038957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346733e-3525-4c9a-8fca-6cd2101a3724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:59.039652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346733e-3525-4c9a-8fca-6cd2101a3724 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.039949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346733e-3525-4c9a-8fca-6cd2101a3724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.040213 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346733e-3525-4c9a-8fca-6cd2101a3724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.040915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346733e-3525-4c9a-8fca-6cd2101a3724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:59.042509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346733e-3525-4c9a-8fca-6cd2101a3724 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:59.054869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1136/2291] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:59.059666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:59.071314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd5c1c-d690-46d2-abfc-b7b43ab2b5f5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:59.076693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:59.077342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.077693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.077996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] PATH_INFO: `/v3/projects/d5897752e4c34a42b85b27f8f877a938/users/737f63c5893c477e89e34c048e1077a5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.093447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5897752e4c34a42b85b27f8f877a938, user_id=737f63c5893c477e89e34c048e1077a5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:59.096611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:59.111344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1156/2292] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:59 2022] PUT /identity/v3/projects/d5897752e4c34a42b85b27f8f877a938/users/737f63c5893c477e89e34c048e1077a5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:59.121637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919b141-d063-4da6-a9fd-7a59d938c6c7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.121934 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919b141-d063-4da6-a9fd-7a59d938c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.122218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919b141-d063-4da6-a9fd-7a59d938c6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.156931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3919b141-d063-4da6-a9fd-7a59d938c6c7 None None] MFA Rules not processed for user `737f63c5893c477e89e34c048e1077a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:59.158206 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3919b141-d063-4da6-a9fd-7a59d938c6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:59.226690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1137/2293] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:59.239074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:59.240820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:59.304253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aca383-2b66-4b27-934f-9e7ba9fe074a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:59.308276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8972fa-506e-4c35-ac8f-bc737b981d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:59.309032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8972fa-506e-4c35-ac8f-bc737b981d39 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.309328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8972fa-506e-4c35-ac8f-bc737b981d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.309724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8972fa-506e-4c35-ac8f-bc737b981d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.327819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8972fa-506e-4c35-ac8f-bc737b981d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:59.331154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8972fa-506e-4c35-ac8f-bc737b981d39 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:59.352094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1157/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:22:59.621474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c11085-9b28-4a04-81ea-c90ff626d51e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.622990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c11085-9b28-4a04-81ea-c90ff626d51e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.623396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c11085-9b28-4a04-81ea-c90ff626d51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.665957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d6c11085-9b28-4a04-81ea-c90ff626d51e None None] MFA Rules not processed for user `0f6938e54a8d49a6b00d3a6694d4e57f`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:59.667445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d6c11085-9b28-4a04-81ea-c90ff626d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:59.751801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:59.771743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1138/2295] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:04.344734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b948e-9a7a-48ca-acde-1b89834d40a1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:04.346628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b948e-9a7a-48ca-acde-1b89834d40a1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:04.347447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b948e-9a7a-48ca-acde-1b89834d40a1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:04.356762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1158/2296] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:23:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:23:04.370006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8c6b-ec51-442d-b85c-1d28351f4790 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:04.370572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8c6b-ec51-442d-b85c-1d28351f4790 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:04.370990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8c6b-ec51-442d-b85c-1d28351f4790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:04.425818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b50d8c6b-ec51-442d-b85c-1d28351f4790 None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:04.427944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b50d8c6b-ec51-442d-b85c-1d28351f4790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:04.575813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1139/2297] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:23:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:04.596856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b948e-9a7a-48ca-acde-1b89834d40a1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:04.598817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-334b948e-9a7a-48ca-acde-1b89834d40a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:04.718292 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b948e-9a7a-48ca-acde-1b89834d40a1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:04.723250 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd47984-35e2-497d-a22c-b55b8833a6a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:04.723737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47984-35e2-497d-a22c-b55b8833a6a0 service glance] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:04.724505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47984-35e2-497d-a22c-b55b8833a6a0 service glance] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:04.724889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47984-35e2-497d-a22c-b55b8833a6a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:04.735921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:04.740847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd47984-35e2-497d-a22c-b55b8833a6a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:04.741936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd47984-35e2-497d-a22c-b55b8833a6a0 service glance] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:04.762880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1159/2298] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:23:04.894170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2da06-96aa-4010-a7b7-e799a6adf6db None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:04.894170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2da06-96aa-4010-a7b7-e799a6adf6db None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:04.894170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2da06-96aa-4010-a7b7-e799a6adf6db None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:04.894170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1140/2299] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:23:04.903652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ca134-0e66-4f33-becb-a6acb5836312 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:04.903966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ca134-0e66-4f33-becb-a6acb5836312 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:04.904225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ca134-0e66-4f33-becb-a6acb5836312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:04.928158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cd8ca134-0e66-4f33-becb-a6acb5836312 None None] MFA Rules not processed for user `771f49a069014fdc956e073c2d0a0fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:04.929432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cd8ca134-0e66-4f33-becb-a6acb5836312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:05.033707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1160/2300] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:23:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:05.063410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2da06-96aa-4010-a7b7-e799a6adf6db None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:05.066701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-41b2da06-96aa-4010-a7b7-e799a6adf6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:05.134394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2da06-96aa-4010-a7b7-e799a6adf6db None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:05.150922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b82eb-6d91-40cc-91d4-53de4fd46c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:05.151721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b82eb-6d91-40cc-91d4-53de4fd46c53 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:05.152003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b82eb-6d91-40cc-91d4-53de4fd46c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:05.155514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b82eb-6d91-40cc-91d4-53de4fd46c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:05.177119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b82eb-6d91-40cc-91d4-53de4fd46c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:05.179371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b82eb-6d91-40cc-91d4-53de4fd46c53 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:05.193987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1141/2301] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:05.410231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd99f48-7a82-49b7-ad54-d43da755e2d7 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:05.410832 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd99f48-7a82-49b7-ad54-d43da755e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:05.410944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd99f48-7a82-49b7-ad54-d43da755e2d7 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:05.413516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1161/2302] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:23:05.421055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:05.421590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:05.421815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:05.479807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] MFA Rules not processed for user `15b5f371cd614e3bb246b015818a97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:05.482524 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:05.497111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1142/2303] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:05.538869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd99f48-7a82-49b7-ad54-d43da755e2d7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:05.540724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8fd99f48-7a82-49b7-ad54-d43da755e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:05.564212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd99f48-7a82-49b7-ad54-d43da755e2d7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:05.568533 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd7a8-8cd4-47f7-8bb0-79676055b46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:05.569551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd7a8-8cd4-47f7-8bb0-79676055b46d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:05.569995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd7a8-8cd4-47f7-8bb0-79676055b46d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:05.570504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd7a8-8cd4-47f7-8bb0-79676055b46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:05.583635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd7a8-8cd4-47f7-8bb0-79676055b46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:05.585727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd7a8-8cd4-47f7-8bb0-79676055b46d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:05.601245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1162/2304] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:11.384590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:11.450916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c0d1e-7d6e-4731-98b6-7e5734ec25ee None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:11.456168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:11.456547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:11.456922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:11.457310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] PATH_INFO: `/v3/users/737f63c5893c477e89e34c048e1077a5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:11.458685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737f63c5893c477e89e34c048e1077a5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:11.460778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:11.529254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737f63c5893c477e89e34c048e1077a5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:11.530252 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737f63c5893c477e89e34c048e1077a5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:11.545939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737f63c5893c477e89e34c048e1077a5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:11.548316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:11.557785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737f63c5893c477e89e34c048e1077a5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:11.558706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737f63c5893c477e89e34c048e1077a5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:11.580487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1143/2305] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:23:11 2022] DELETE /identity/v3/users/737f63c5893c477e89e34c048e1077a5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:11.586696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:11.587628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:11.587628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:11.610308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:11.611634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:11.679825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1163/2306] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:11.690074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:11.691627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:11.760059 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c2ed9-4463-40b4-bb1f-7a9250a50682 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:11.764397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a476bac-1f62-49de-90c2-63182cc3f277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:11.765977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a476bac-1f62-49de-90c2-63182cc3f277 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:11.766410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a476bac-1f62-49de-90c2-63182cc3f277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:11.766959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a476bac-1f62-49de-90c2-63182cc3f277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:11.779041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a476bac-1f62-49de-90c2-63182cc3f277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:11.780609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a476bac-1f62-49de-90c2-63182cc3f277 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:11.791740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1144/2307] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:12.639011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:12.641865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:12.666121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcfca3-394c-45a3-bddb-5323ef9cf01f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:12.676481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:12.677175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:12.677458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:12.679797 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] PATH_INFO: `/v3/projects/d5897752e4c34a42b85b27f8f877a938` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:12.682828 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5897752e4c34a42b85b27f8f877a938)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:12.687280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:12.800251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5897752e4c34a42b85b27f8f877a938', 'request_id': 'req-0dafd796-ce3c-4145-8d2f-b3203ef9481c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:12.809321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5897752e4c34a42b85b27f8f877a938', 'request_id': 'req-0dafd796-ce3c-4145-8d2f-b3203ef9481c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:12.822053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5897752e4c34a42b85b27f8f877a938', 'request_id': 'req-0dafd796-ce3c-4145-8d2f-b3203ef9481c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:12.849071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] The token cache is being invalidate because project d5897752e4c34a42b85b27f8f877a938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:23:12.850037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:12.851546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0dafd796-ce3c-4145-8d2f-b3203ef9481c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:12.854463 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1164/2308] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:23:12 2022] DELETE /identity/v3/projects/d5897752e4c34a42b85b27f8f877a938 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:12.861235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:12.862598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:12.863449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:12.902115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:12.904112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:12.997468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1145/2309] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:13.004030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.004367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.004672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.028116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:13.029583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:13.042557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1165/2310] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:13.043311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.053957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe557f5-7afa-418e-bba0-fbc4c1accdf8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.060769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.062146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.062385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.062641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.063396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:23:13.063796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.065761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.083727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1146/2311] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:23:13.085309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.097743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab08ce5-87e6-4c3a-a9ff-46d8060a9464 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.104437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.105153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.105522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.105799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.106679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.109272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.115687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:13.152228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1166/2312] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:23:13 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:23:13.156413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.176579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6fd2f-478c-4e8c-b310-77e24c66b0a3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.304224 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.305987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.306889 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.307673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.310433 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.314703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.359396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1147/2313] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:23:13 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:23:13.363106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.381876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a44c1-6816-4e07-b642-46b4a9b002bc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.388110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.389279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.389687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.389909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.390638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.392372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.407288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1167/2314] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:23:13.408895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.415245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:13.423366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea947-af8f-48c4-847f-0605275d19a5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.435382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e63b1-874d-4fb2-8b6e-04f711a262c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.436212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e63b1-874d-4fb2-8b6e-04f711a262c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.436559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e63b1-874d-4fb2-8b6e-04f711a262c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.436845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e63b1-874d-4fb2-8b6e-04f711a262c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.438129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e63b1-874d-4fb2-8b6e-04f711a262c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.439864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e63b1-874d-4fb2-8b6e-04f711a262c2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.461826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1148/2315] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:23:13.468556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.487014 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d399c-fe59-4e4b-a09c-8be00f529068 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.492454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.493254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.493610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.493963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] PATH_INFO: `/v3/projects/6eacbee63bfa4aaba7cb0025db3bc335/users/5865f1671bf14d57980b7036ae7a16f0/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.516390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eacbee63bfa4aaba7cb0025db3bc335, user_id=5865f1671bf14d57980b7036ae7a16f0, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.519161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.539068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1168/2316] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:23:13 2022] PUT /identity/v3/projects/6eacbee63bfa4aaba7cb0025db3bc335/users/5865f1671bf14d57980b7036ae7a16f0/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:13.549807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf802f2-1217-4e35-af0d-12009a5c5f83 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.550073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf802f2-1217-4e35-af0d-12009a5c5f83 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.550402 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf802f2-1217-4e35-af0d-12009a5c5f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.581385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-dcf802f2-1217-4e35-af0d-12009a5c5f83 None None] MFA Rules not processed for user `5865f1671bf14d57980b7036ae7a16f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:13.582629 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dcf802f2-1217-4e35-af0d-12009a5c5f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:13.654052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1149/2317] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:13.667841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:13.669929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:13.729559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea11b77-ed18-4aa6-8864-18055c146e05 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:13.733294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:13.734484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:13.734769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:13.735049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:13.744820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:13.746266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:13.756754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1169/2318] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:15.145582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:15.146126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:15.146126 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:15.167807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:15.172164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:15.295771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1150/2319] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:15.311496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:15.313774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:15.457368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb036-1062-4a4f-bbec-671ca2f445df service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:15.464482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229325-1e6d-4bdd-b484-fe6a905ecebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:15.465261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229325-1e6d-4bdd-b484-fe6a905ecebf service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:15.465624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229325-1e6d-4bdd-b484-fe6a905ecebf service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:15.466596 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229325-1e6d-4bdd-b484-fe6a905ecebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:15.478803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f229325-1e6d-4bdd-b484-fe6a905ecebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:15.480580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f229325-1e6d-4bdd-b484-fe6a905ecebf service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:15.491224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1170/2320] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:23:25.271323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=61 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:25.487565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=78,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:34.036180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:34.107636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a492b-4c55-44c3-a5e1-3030cd3d6309 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:34.112453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:34.113095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:34.113371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:34.113784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] PATH_INFO: `/v3/users/5865f1671bf14d57980b7036ae7a16f0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:34.114965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5865f1671bf14d57980b7036ae7a16f0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:34.117005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:34.178448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5865f1671bf14d57980b7036ae7a16f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.179129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5865f1671bf14d57980b7036ae7a16f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.204187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5865f1671bf14d57980b7036ae7a16f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.211870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5865f1671bf14d57980b7036ae7a16f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.212687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5865f1671bf14d57980b7036ae7a16f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.228435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1151/2321] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:23:34 2022] DELETE /identity/v3/users/5865f1671bf14d57980b7036ae7a16f0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:34.233720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:34.234662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:34.234662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:34.258177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:34.259409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:34.346338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1171/2322] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:23:34.357296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:34.360194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:34.427771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7085-68ba-4b2b-886b-c64006964813 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:34.431478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:34.432107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:34.432393 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:34.432700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:34.442461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:34.443936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:34.453401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1152/2323] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:34.756583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:34.758077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:34.783731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217eee1-41b8-4686-beb8-172cf8ebf0bc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:34.791770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:34.792439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:34.792741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:34.793062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] PATH_INFO: `/v3/projects/6eacbee63bfa4aaba7cb0025db3bc335` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:34.794274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eacbee63bfa4aaba7cb0025db3bc335)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:34.796318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:34.848638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eacbee63bfa4aaba7cb0025db3bc335', 'request_id': 'req-9f007def-e2ba-4aba-a07e-77b1f3da4396'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.854890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eacbee63bfa4aaba7cb0025db3bc335', 'request_id': 'req-9f007def-e2ba-4aba-a07e-77b1f3da4396'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.862636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eacbee63bfa4aaba7cb0025db3bc335', 'request_id': 'req-9f007def-e2ba-4aba-a07e-77b1f3da4396'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.875095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] The token cache is being invalidate because project 6eacbee63bfa4aaba7cb0025db3bc335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:23:34.875372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.876110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9f007def-e2ba-4aba-a07e-77b1f3da4396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:34.886811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1172/2324] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:23:34 2022] DELETE /identity/v3/projects/6eacbee63bfa4aaba7cb0025db3bc335 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:23:41.011790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:41.015930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:41.049769 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:41.104977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e8692-8fd4-48cb-9215-463aedc5797c service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:41.181756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:41.182335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:41.182628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:41.182909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:41.184805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:41.201646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:41.203847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:41.223239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1153/2325] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:42.592418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2033-ca67-41f6-84c4-717ee5fc442b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:42.593052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2033-ca67-41f6-84c4-717ee5fc442b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:42.593052 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2033-ca67-41f6-84c4-717ee5fc442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:42.618890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-111c2033-ca67-41f6-84c4-717ee5fc442b None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:42.620548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-111c2033-ca67-41f6-84c4-717ee5fc442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:42.709764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1173/2326] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:23:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:23:42.739112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:42.740607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:42.817058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c46b1-9328-4bd5-980c-f11ccbf85931 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:42.825263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:42.826016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:42.826350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:42.826648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:42.836352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:42.838020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:42.849021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1154/2327] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:43.443190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429f86-4d56-44fe-a691-3863cd7c0827 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:43.443540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429f86-4d56-44fe-a691-3863cd7c0827 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:43.443770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429f86-4d56-44fe-a691-3863cd7c0827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:43.463763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bd429f86-4d56-44fe-a691-3863cd7c0827 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:43.465859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bd429f86-4d56-44fe-a691-3863cd7c0827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:43.473589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1174/2328] 10.222.0.17 () {58 vars in 967 bytes} [Mon Aug 1 14:23:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:23:43.486168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:43.497129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff9720-2f97-4d2b-9002-88678aeb097e service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:43.502616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:43.503323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:43.503703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:43.504072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:43.513123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:43.515377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:43.524914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1155/2329] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:43.782508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a45cff-390a-4579-be32-5eaef92589e8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:43.782992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a45cff-390a-4579-be32-5eaef92589e8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:43.783096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a45cff-390a-4579-be32-5eaef92589e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:43.803152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-33a45cff-390a-4579-be32-5eaef92589e8 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:43.804656 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-33a45cff-390a-4579-be32-5eaef92589e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:43.814385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1175/2330] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:23:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:43.830128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:43.841870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e125a2-e9b7-464d-96a9-7865eec9e306 service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:43.847680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:43.848278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:43.848556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:43.848832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:43.860427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:43.862082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:43.871483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1156/2331] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:44.315431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857d0ee-a279-41ad-8351-87b2ff686b68 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:44.315758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857d0ee-a279-41ad-8351-87b2ff686b68 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:44.316168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857d0ee-a279-41ad-8351-87b2ff686b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:44.342011 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8857d0ee-a279-41ad-8351-87b2ff686b68 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:44.344066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8857d0ee-a279-41ad-8351-87b2ff686b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:44.353161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1176/2332] 10.222.0.17 () {58 vars in 967 bytes} [Mon Aug 1 14:23:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:44.367561 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:44.385149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631ba8b-bc35-435b-94d7-f9e5641cb7de service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:44.390968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:44.391951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:44.392274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:44.392658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:44.402893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:44.404553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:44.415177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1157/2333] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:49.697627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:49.698136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:49.698235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:49.717630 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:49.718841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:49.728607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1177/2334] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:49.740569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:49.747349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:49.752585 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ca63-d23f-4b8c-aa90-5eb73a0a0fc0 service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:49.762646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27e02a-f6e1-4dbb-98d2-3f6b13508d7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:49.763337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e02a-f6e1-4dbb-98d2-3f6b13508d7c service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:49.763857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e02a-f6e1-4dbb-98d2-3f6b13508d7c service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:49.764144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e02a-f6e1-4dbb-98d2-3f6b13508d7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:49.774261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27e02a-f6e1-4dbb-98d2-3f6b13508d7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:49.775932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27e02a-f6e1-4dbb-98d2-3f6b13508d7c service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:49.785743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1158/2335] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:49.832266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:49.833974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:49.839366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:49.949859 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15923754-5944-4a1d-9ce2-e94d6ce8f316 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:49.954819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:49.955493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:49.955778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:49.956059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] PATH_INFO: `/v3/users/a5711f44f73e4925ae52a784f43a2a1c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:49.957261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5711f44f73e4925ae52a784f43a2a1c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:49.959502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:50.015640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5711f44f73e4925ae52a784f43a2a1c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.016300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5711f44f73e4925ae52a784f43a2a1c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.029918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5711f44f73e4925ae52a784f43a2a1c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.041258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5711f44f73e4925ae52a784f43a2a1c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.049893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5711f44f73e4925ae52a784f43a2a1c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.056112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1178/2336] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:23:49 2022] DELETE /identity/v3/users/a5711f44f73e4925ae52a784f43a2a1c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:50.060859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:50.061311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:50.062121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:50.084352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:50.085988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:50.175188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1159/2337] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:50.187311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:50.188945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:50.260309 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f0cb-f771-4b8f-852c-9f712704a847 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:50.264384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075639d-33c3-4284-93ac-5db3a6202c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:50.265372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075639d-33c3-4284-93ac-5db3a6202c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:50.265841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075639d-33c3-4284-93ac-5db3a6202c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:50.266301 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075639d-33c3-4284-93ac-5db3a6202c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:50.276982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075639d-33c3-4284-93ac-5db3a6202c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:50.279151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075639d-33c3-4284-93ac-5db3a6202c88 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:50.314796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1179/2338] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:50.724770 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:50.726774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:50.766072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f3a99-b5e0-412f-abb7-4a75e749df59 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:50.772907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:50.773885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:50.774443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:50.774914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] PATH_INFO: `/v3/projects/27f6ab4a05704f8c97a5e1883becb247` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:50.776396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f6ab4a05704f8c97a5e1883becb247)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:50.781110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:50.854129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f6ab4a05704f8c97a5e1883becb247', 'request_id': 'req-c6cd784e-17bc-44bf-bc27-2520cabdb72a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.864267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f6ab4a05704f8c97a5e1883becb247', 'request_id': 'req-c6cd784e-17bc-44bf-bc27-2520cabdb72a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.881647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f6ab4a05704f8c97a5e1883becb247', 'request_id': 'req-c6cd784e-17bc-44bf-bc27-2520cabdb72a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.917067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] The token cache is being invalidate because project 27f6ab4a05704f8c97a5e1883becb247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:23:50.918635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.936349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c6cd784e-17bc-44bf-bc27-2520cabdb72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:50.942794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1160/2339] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:23:50 2022] DELETE /identity/v3/projects/27f6ab4a05704f8c97a5e1883becb247 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:53.759289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:53.759845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:53.759947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:53.840947 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] MFA Rules not processed for user `c487e443eb274090bce842e855b015ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:53.845611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:53.876066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:53.934148 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1180/2340] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:56.740977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd149-9456-4e6b-b3ab-0dc072a3c3f6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:56.743332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd149-9456-4e6b-b3ab-0dc072a3c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:56.743820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd149-9456-4e6b-b3ab-0dc072a3c3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:56.773959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-52ecd149-9456-4e6b-b3ab-0dc072a3c3f6 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:56.775216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-52ecd149-9456-4e6b-b3ab-0dc072a3c3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:56.928618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1161/2341] 10.222.0.17 () {58 vars in 967 bytes} [Mon Aug 1 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:56.947916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:56.950095 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:57.078677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f6227-708b-4d65-a444-2a45b695f50e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:57.089545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:57.090189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:57.090672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:57.090783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:57.112190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:57.113804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:57.129203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1181/2342] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:24:06.872129 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:07.120950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=46 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:09.250652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:09.251120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:09.251211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:09.271292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:09.272544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:09.280437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1162/2343] 10.222.0.17 () {58 vars in 967 bytes} [Mon Aug 1 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:09.292852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:09.304334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bc3cb-2462-412d-9cf0-dd03383e82dd service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:09.309950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:09.310587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:09.310873 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:09.311150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:09.321413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:09.323210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:09.333760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1182/2344] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:24:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:24:11.520961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:11.522600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:11.603109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc91e55-f38d-43bd-bdbd-7e9e0737ac5e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:11.607949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:11.609209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:11.609487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:11.609784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] PATH_INFO: `/v3/users/bbbdf80e14b647cbbdb3d292d534d987` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:11.611013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbdf80e14b647cbbdb3d292d534d987)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:11.613038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:11.670173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbdf80e14b647cbbdb3d292d534d987'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:11.670881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbdf80e14b647cbbdb3d292d534d987'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:11.688263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbdf80e14b647cbbdb3d292d534d987'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:11.700816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbdf80e14b647cbbdb3d292d534d987'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:11.702424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5a222577-1ada-4f05-bdd0-712d311098a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbdf80e14b647cbbdb3d292d534d987'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:11.724609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1163/2345] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:24:11 2022] DELETE /identity/v3/users/bbbdf80e14b647cbbdb3d292d534d987 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:11.926103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:11.943769 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444906-721d-4c73-8418-b3d9a493fece service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:12.029082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:12.030514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:12.031171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:12.031171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] PATH_INFO: `/v3/projects/d6dd5eef6c62467ab15a23233e366185` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:12.032775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6dd5eef6c62467ab15a23233e366185)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:12.035297 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:12.097461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6dd5eef6c62467ab15a23233e366185', 'request_id': 'req-4f67c27f-16ab-4144-8df1-029d4a3ef2da'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:12.113484 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6dd5eef6c62467ab15a23233e366185', 'request_id': 'req-4f67c27f-16ab-4144-8df1-029d4a3ef2da'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:12.122666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6dd5eef6c62467ab15a23233e366185', 'request_id': 'req-4f67c27f-16ab-4144-8df1-029d4a3ef2da'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:12.134883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] The token cache is being invalidate because project d6dd5eef6c62467ab15a23233e366185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:24:12.135224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:12.136007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4f67c27f-16ab-4144-8df1-029d4a3ef2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:12.139530 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1183/2346] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:24:11 2022] DELETE /identity/v3/projects/d6dd5eef6c62467ab15a23233e366185 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:17.640815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabf57d-3306-48cf-979b-2071a37fe38c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:17.641304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabf57d-3306-48cf-979b-2071a37fe38c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:17.641446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabf57d-3306-48cf-979b-2071a37fe38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:17.662698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0cabf57d-3306-48cf-979b-2071a37fe38c None None] MFA Rules not processed for user `9eda4e19122444cf9cb2b6941adff9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:17.663949 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0cabf57d-3306-48cf-979b-2071a37fe38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:17.719257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1164/2347] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:22.131798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:27.711875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:37.319080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da86c-c7e9-41d5-a3db-bdddf01d42e1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:37.319791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da86c-c7e9-41d5-a3db-bdddf01d42e1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:37.319791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da86c-c7e9-41d5-a3db-bdddf01d42e1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:37.321189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1184/2348] 10.222.0.17 () {54 vars in 921 bytes} [Mon Aug 1 14:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:24:37.354824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:37.355403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:37.355601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:37.376761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] MFA Rules not processed for user `324fc1c2ea18420eb01d8b65683818ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:37.378098 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:37.399112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da86c-c7e9-41d5-a3db-bdddf01d42e1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:37.402399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-db6da86c-c7e9-41d5-a3db-bdddf01d42e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:37.452101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1165/2349] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Aug 1 14:24:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:37.464468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:37.466092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:37.475741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da86c-c7e9-41d5-a3db-bdddf01d42e1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:37.479085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62641366-a34b-40f4-ace5-0e5ce7fa1a13 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:37.480755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:37.481411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:37.481748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:37.482028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] PATH_INFO: `/v3/users/0f6938e54a8d49a6b00d3a6694d4e57f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:37.483220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6938e54a8d49a6b00d3a6694d4e57f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:37.484374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:37.485015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:37.485213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:37.485332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:37.485562 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:37.494419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:37.496016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:37.507713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1166/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:24:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:24:37.537581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6938e54a8d49a6b00d3a6694d4e57f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.538278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6938e54a8d49a6b00d3a6694d4e57f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.550896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6938e54a8d49a6b00d3a6694d4e57f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.558197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6938e54a8d49a6b00d3a6694d4e57f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.566170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b96f2f58-04be-4beb-afb9-c56126239b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6938e54a8d49a6b00d3a6694d4e57f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.572151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1185/2351] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:24:37 2022] DELETE /identity/v3/users/0f6938e54a8d49a6b00d3a6694d4e57f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:37.749168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:37.770809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f5e8d-f018-49a5-acf0-3baa9acfac20 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:37.829276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:37.829926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:37.830210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:37.830495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] PATH_INFO: `/v3/projects/486a69a51709439aad36192aece6133e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:37.831696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486a69a51709439aad36192aece6133e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:37.833663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:37.881317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486a69a51709439aad36192aece6133e', 'request_id': 'req-b2ef757c-5b70-4597-8a4f-470fd382a2f4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.886803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486a69a51709439aad36192aece6133e', 'request_id': 'req-b2ef757c-5b70-4597-8a4f-470fd382a2f4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.893999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486a69a51709439aad36192aece6133e', 'request_id': 'req-b2ef757c-5b70-4597-8a4f-470fd382a2f4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.904961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] The token cache is being invalidate because project 486a69a51709439aad36192aece6133e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:24:37.905246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.905871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2ef757c-5b70-4597-8a4f-470fd382a2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:37.907371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1167/2352] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:24:37 2022] DELETE /identity/v3/projects/486a69a51709439aad36192aece6133e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:37.912437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:37.912796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:37.913111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:37.933425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:37.934643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:37.994266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1186/2353] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:24:38.002893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.003201 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.003505 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.023043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:38.024308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:38.032345 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1168/2354] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:38.036524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.046365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f029eb6-b825-452f-b9f2-a0a764ad4437 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.051518 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.052124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.052408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.052685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.053416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:24:38.053722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.055324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.067205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1187/2355] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:24:38.071525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.081362 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd92a6b-027a-471e-959b-6d5758c66ba4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.085963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.086576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.086861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.087137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.087865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.089758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.095148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:38.123431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1169/2356] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:24:38 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:24:38.127654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.137834 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4caf05-7b92-467f-ad45-6b5039f3962d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.142176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:38.197164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.197777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.198061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.198335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.198995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.200952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.222657 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1188/2357] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:24:38 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:24:38.226772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.237169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a0ad0-330c-40a5-95cb-ed23fb083ac0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.241710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.242315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.242597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.243005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.243729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.245322 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.256150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1170/2358] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:24:38.260276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.271004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfa7f5-f3aa-4f42-8bd4-d767682d5694 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.276066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e58e5-5c99-4975-b760-05a06687f103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.276677 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e58e5-5c99-4975-b760-05a06687f103 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.276969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e58e5-5c99-4975-b760-05a06687f103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.277265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e58e5-5c99-4975-b760-05a06687f103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.277965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e58e5-5c99-4975-b760-05a06687f103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.279644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e58e5-5c99-4975-b760-05a06687f103 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.290600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1189/2359] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:24:38.294746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.305087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af419a3-d7a1-4326-9fc1-6713b2336e17 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.309811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.310471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.310759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.311040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] PATH_INFO: `/v3/projects/d100a3378465426ebda66ca171e204d5/users/dd4b19d434e144d99d9ce459acabd538/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.326299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d100a3378465426ebda66ca171e204d5, user_id=dd4b19d434e144d99d9ce459acabd538, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.329647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.343313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1171/2360] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:24:38 2022] PUT /identity/v3/projects/d100a3378465426ebda66ca171e204d5/users/dd4b19d434e144d99d9ce459acabd538/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:38.355129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.355635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.355730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.383861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] MFA Rules not processed for user `dd4b19d434e144d99d9ce459acabd538`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:38.385084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:38.448210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1190/2361] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:38.460172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:38.461903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:38.521998 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e00c4-69e0-4f87-bbe1-20e233ac2de5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:38.525633 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926ac6-d7bb-4cc4-a246-62082c95f9e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:38.526746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926ac6-d7bb-4cc4-a246-62082c95f9e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:38.526746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926ac6-d7bb-4cc4-a246-62082c95f9e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:38.526973 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926ac6-d7bb-4cc4-a246-62082c95f9e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:38.535232 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02926ac6-d7bb-4cc4-a246-62082c95f9e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:38.536700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02926ac6-d7bb-4cc4-a246-62082c95f9e7 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:38.545459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1172/2362] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:24:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:24:46.612972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:46.676969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7e4a6-0fe7-4dfb-8a7e-48538f878bd0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:46.681476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:46.682111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:46.682393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:46.682669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] PATH_INFO: `/v3/users/dd4b19d434e144d99d9ce459acabd538` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:46.683891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4b19d434e144d99d9ce459acabd538)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:46.685815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:46.701943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:46.733422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4b19d434e144d99d9ce459acabd538'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:46.734138 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4b19d434e144d99d9ce459acabd538'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:46.746084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4b19d434e144d99d9ce459acabd538'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:46.754541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4b19d434e144d99d9ce459acabd538'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:46.761861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4b19d434e144d99d9ce459acabd538'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:46.768453 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1191/2363] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:24:46 2022] DELETE /identity/v3/users/dd4b19d434e144d99d9ce459acabd538 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:46.772987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:46.773452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:46.773567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:46.793758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:46.794517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:46.855526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1173/2364] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:46.866133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:46.867565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:46.926387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2795afc-d7c3-4e89-b8bb-7464029f1d79 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:46.930153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f655d5-dbff-4575-8f01-e6ac251b66d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:46.930854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f655d5-dbff-4575-8f01-e6ac251b66d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:46.931041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f655d5-dbff-4575-8f01-e6ac251b66d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:46.931311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f655d5-dbff-4575-8f01-e6ac251b66d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:46.940961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f655d5-dbff-4575-8f01-e6ac251b66d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:46.942374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f655d5-dbff-4575-8f01-e6ac251b66d2 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:46.951986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1192/2365] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:24:47.230255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.231750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:47.245620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c56df-d147-41ba-92ee-d4cd7ec1bdb8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.251271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.251960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.252242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.252518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] PATH_INFO: `/v3/projects/d100a3378465426ebda66ca171e204d5` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.254200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d100a3378465426ebda66ca171e204d5)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.259220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.310191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:47.311549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd100a3378465426ebda66ca171e204d5', 'request_id': 'req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:47.318501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd100a3378465426ebda66ca171e204d5', 'request_id': 'req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:47.326979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd100a3378465426ebda66ca171e204d5', 'request_id': 'req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:47.339050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] The token cache is being invalidate because project d100a3378465426ebda66ca171e204d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:24:47.339344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:47.340012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-134d08bb-82fb-4f2a-a7ec-a49a430e86b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:24:47.341655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1174/2366] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:24:47 2022] DELETE /identity/v3/projects/d100a3378465426ebda66ca171e204d5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:47.347560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.347855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.348136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.368411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:47.369587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:47.432356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1193/2367] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:24:47.442357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.442654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.442936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.476692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:47.478250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:47.490368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1175/2368] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:47.498075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.508433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8caa4-aa8f-4ed2-859f-2ccb3caef5b7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.515664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.516310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.516597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.516874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.518232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:24:47.518232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.520119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.537833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1194/2369] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:24:47.539197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.549913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92d728-c730-43a6-9417-16ef2d6e1fbc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.555071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.555684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.555972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.556248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.556927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.559071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.596895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1176/2370] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:24:47 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:24:47.598188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.609184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2038-41f5-467e-954a-4a44632baf45 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.633969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:47.688743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.689393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.689786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.690170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.691001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.693154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.717337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1195/2371] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:24:47 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:24:47.722840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.734326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23fce6-516f-4959-b25d-bcf878913857 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.739521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.740380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.740666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.740974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.741688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.743317 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.757415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1177/2372] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:24:47.758743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.770145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614228c3-ec87-408e-83ee-184506bd28b7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.774641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db7dd-94e6-488c-b4b3-710f1252c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.775262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db7dd-94e6-488c-b4b3-710f1252c740 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.775572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db7dd-94e6-488c-b4b3-710f1252c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.775850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db7dd-94e6-488c-b4b3-710f1252c740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.776550 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db7dd-94e6-488c-b4b3-710f1252c740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.778150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db7dd-94e6-488c-b4b3-710f1252c740 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.790038 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1196/2373] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:24:47.794310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.804748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64ef7-c09e-4306-afc2-87de090f4f92 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:47.809756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:47.810587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.810912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.811149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] PATH_INFO: `/v3/projects/83c251299d274471b80df4ff026b0102/users/00393da6a11e4d77b5ad25b904f7187b/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.826381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c251299d274471b80df4ff026b0102, user_id=00393da6a11e4d77b5ad25b904f7187b, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:47.829602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:47.845501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1178/2374] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:24:47 2022] PUT /identity/v3/projects/83c251299d274471b80df4ff026b0102/users/00393da6a11e4d77b5ad25b904f7187b/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:24:47.850750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39f16c-2647-4fa2-8bd3-956edff9aaa3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:47.851116 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39f16c-2647-4fa2-8bd3-956edff9aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:47.851475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39f16c-2647-4fa2-8bd3-956edff9aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:47.871308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6f39f16c-2647-4fa2-8bd3-956edff9aaa3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:47.872892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6f39f16c-2647-4fa2-8bd3-956edff9aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:47.940290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1197/2375] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:24:47.949997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:47.952120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:48.011071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d436-4a9c-43e7-92ef-a87e5d74b23a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:48.014857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac85e4-bda3-4afb-be51-dec71ca147e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:48.015500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac85e4-bda3-4afb-be51-dec71ca147e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:48.015855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac85e4-bda3-4afb-be51-dec71ca147e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:48.016637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac85e4-bda3-4afb-be51-dec71ca147e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:48.026093 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac85e4-bda3-4afb-be51-dec71ca147e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:48.027564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac85e4-bda3-4afb-be51-dec71ca147e0 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:48.037227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1179/2376] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:24:52.302240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:52.303019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:52.303019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:52.322609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] MFA Rules not processed for user `fbaa67d901e34419b1616a8457fbd683`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:52.324249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:52.415612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1198/2377] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:52.538409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:52.538856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:52.539032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:52.568777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] MFA Rules not processed for user `00393da6a11e4d77b5ad25b904f7187b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:52.570277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:52.679273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1180/2378] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:52.688272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:52.703096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c3668-bcf6-4bb8-9f54-6963c513d3cd None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:52.708883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:52.709488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:52.709778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:52.710056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:52.710770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:52.712858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:52.759826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1199/2379] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:24:52 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:24:52.761036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:52.770403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:52.776776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c403f43-c5c5-4a1f-804d-149e5c179730 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:52.870753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:52.871411 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:52.871738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:52.872529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:52.873231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:52.876138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:52.902698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1181/2380] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:24:52 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:24:52.904356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:52.916791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889a293-c688-4cac-82db-2f175c203fdf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:52.922655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:52.924008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:52.924506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:52.925015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:52.925941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:52.927853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:52.942818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1200/2381] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:24:52.946715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:52.966432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fcb8c-d3f1-4126-a6d6-02394f97766b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:52.972781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:52.973478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:52.973762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:52.974040 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] PATH_INFO: `/v3/projects/18402d0b12754d42a679dc9a70398e9b/users/55d498154b384496b6e0fe95511096f6/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:53.004795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18402d0b12754d42a679dc9a70398e9b, user_id=55d498154b384496b6e0fe95511096f6, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:53.009184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:53.028900 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1182/2382] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:24:52 2022] PUT /identity/v3/projects/18402d0b12754d42a679dc9a70398e9b/users/55d498154b384496b6e0fe95511096f6/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:24:53.035803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:53.046252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de438cc-8dfc-4f82-87fb-e078f056ec90 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:53.063033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:53.130995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfe42b-177d-4d75-b32f-3a41593aa2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:53.131662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfe42b-177d-4d75-b32f-3a41593aa2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:53.132008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfe42b-177d-4d75-b32f-3a41593aa2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:53.132221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfe42b-177d-4d75-b32f-3a41593aa2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:53.132930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfe42b-177d-4d75-b32f-3a41593aa2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:53.134534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfe42b-177d-4d75-b32f-3a41593aa2d0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:53.146461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1201/2383] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:24:53.150850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:53.166823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaa157-406f-4a27-9a18-a16121fce7eb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:53.175620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:53.176205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:53.176497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:53.176930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] PATH_INFO: `/v3/projects/18402d0b12754d42a679dc9a70398e9b/users/55d498154b384496b6e0fe95511096f6/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:53.178952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18402d0b12754d42a679dc9a70398e9b, user_id=55d498154b384496b6e0fe95511096f6, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:53.194186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:53.217114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1183/2384] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:24:53 2022] PUT /identity/v3/projects/18402d0b12754d42a679dc9a70398e9b/users/55d498154b384496b6e0fe95511096f6/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:59.032223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1947a1-4074-4ed4-9eef-63daf959e8d9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:59.032734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1947a1-4074-4ed4-9eef-63daf959e8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:59.032838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1947a1-4074-4ed4-9eef-63daf959e8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:59.080847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6e1947a1-4074-4ed4-9eef-63daf959e8d9 None None] MFA Rules not processed for user `55d498154b384496b6e0fe95511096f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:59.082087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6e1947a1-4074-4ed4-9eef-63daf959e8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:59.212032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1202/2385] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:24:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:59.227223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:59.229269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:59.328830 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c84410-f61a-4c7d-90cf-7fc090a006b8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:59.332818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee96f24e85de4092b31081d44fe0a03d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:59.333544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:59.334120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:59.334635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:59.407438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:59.408992 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:59.422894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1184/2386] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:24:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:24:59.599880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7920c-afd6-4c6d-b3c7-817713dcf688 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:59.601143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7920c-afd6-4c6d-b3c7-817713dcf688 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:59.601433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7920c-afd6-4c6d-b3c7-817713dcf688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:59.639160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ebc7920c-afd6-4c6d-b3c7-817713dcf688 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:24:59.646046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ebc7920c-afd6-4c6d-b3c7-817713dcf688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:59.713449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1203/2387] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:24:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:24:59.738109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:24:59.739718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:24:59.754634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:59.820777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce23db-a33c-45fa-bef7-e2e6bd4a6e3d service glance] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:24:59.825822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714c9c0-48de-445e-b03c-eb09c166e420 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:24:59.826602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714c9c0-48de-445e-b03c-eb09c166e420 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:24:59.826887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714c9c0-48de-445e-b03c-eb09c166e420 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:24:59.827187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714c9c0-48de-445e-b03c-eb09c166e420 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:24:59.836557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714c9c0-48de-445e-b03c-eb09c166e420 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:24:59.838225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714c9c0-48de-445e-b03c-eb09c166e420 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:24:59.847685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1185/2388] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:24:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:25:09.144941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:09.145428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:09.145528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:09.165316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] MFA Rules not processed for user `00393da6a11e4d77b5ad25b904f7187b`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:25:09.166558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:09.174335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1204/2389] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:25:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:25:09.843635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=27 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:25:11.893495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:11.894112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:11.894112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:11.915132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:25:11.916398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:11.924329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1186/2390] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:25:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:25:11.936674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:11.949359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779d9bc-b2b1-4a44-bb70-8b7ebbecd080 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:11.954537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:11.955197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:11.955505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:11.955787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:11.966792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:11.969168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:11.988972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1205/2391] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:25:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:25:13.045140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:13.047270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:13.110158 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07206c42-f333-4242-95f7-9bd32fa63939 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:13.113882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:13.114642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:13.114922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:13.115296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:13.117299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:13.206247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:13.207915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:13.222299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1187/2392] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:14.888738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:14.899114 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198f7db-b25f-4dc9-8145-e989b0e4f320 service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:14.902736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:14.903507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:14.903798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:14.904073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:14.905815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:14.925862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:14.927477 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:14.941101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1206/2393] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:21.376723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:21.378333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:21.435189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b066-24d2-4771-997e-0d2049331c4c service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:21.439656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:21.440259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:21.440539 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:21.440813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:21.442550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:21.492502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:25:21.507337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:21.508993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcdde0-c5ff-44cb-a294-fe7c9a1b2aaa service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:21.520686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1188/2394] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:24.934351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:25:31.515372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:25:33.784049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:33.793891 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb703e8d-6210-407d-9513-f5ac78ce95bd service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:33.798840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106e35-6482-4924-878f-880573df54ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:33.799737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106e35-6482-4924-878f-880573df54ee service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:33.800124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106e35-6482-4924-878f-880573df54ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:33.800526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106e35-6482-4924-878f-880573df54ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:33.802858 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-48106e35-6482-4924-878f-880573df54ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:33.814687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106e35-6482-4924-878f-880573df54ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:33.820254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106e35-6482-4924-878f-880573df54ee service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:33.832819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1207/2395] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:36.150840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:36.151395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:36.151395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:36.170946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] MFA Rules not processed for user `c487e443eb274090bce842e855b015ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:25:36.172249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:36.237683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1189/2396] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:25:38.228557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:38.229064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:38.229295 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:38.250020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] MFA Rules not processed for user `9eda4e19122444cf9cb2b6941adff9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:25:38.251243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:38.258468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1208/2397] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:25:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:25:38.560011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:38.570494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cf04a-4ce9-4d77-81bf-b8b4c0714bfb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:38.575267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:38.576351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:38.576641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:38.576920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:38.578837 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:38.590544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:38.592030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:38.601631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1190/2398] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:38.924066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:38.925465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:39.008555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf008-29d8-464a-8cbd-fe59c426fce1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:39.013520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:39.014140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:39.014416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:39.014692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:39.016632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:39.028425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:39.029932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:39.040476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1209/2399] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:42.466228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:42.477482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5a8ef-da22-4fcf-ba86-d147880f6d94 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:42.481597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:42.482360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:42.482799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:42.483181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:42.493300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:42.494972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:42.507695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1191/2400] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:42.805203 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:42.808136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:42.830560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbba00-b5d4-4fd7-90df-25b68d88c0ee service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:42.839117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:42.839796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:42.840638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:42.845154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:42.849486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:42.865680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:42.867790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:42.881105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1210/2401] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:25:47.868508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:47.880906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe182e-1ce2-48d3-af07-00012d84477e service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:47.886404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:47.887020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:47.887308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:47.887628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:47.890368 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:47.902752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:47.904440 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f42872-4e1e-4ee1-977c-315b31654ea3 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:47.915986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1192/2402] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:25:52.238387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:25:52.250610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c4ab2-2d9b-4113-891f-77861e2bb530 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:25:52.257233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:25:52.257863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:25:52.258143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:25:52.258420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:25:52.260322 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:25:52.334726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:25:52.336396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:25:52.347084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1211/2403] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:25:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:25:57.911364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=43 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:02.341602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=75 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:08.105910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:08.106443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:08.106545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:08.127690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:26:08.128924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:08.136635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1193/2404] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:26:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:26:08.159286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:08.170631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c16327-da7b-40f5-bcd1-49946bd13203 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:08.177271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09144686-2dcd-4117-8f87-23aab2797fe9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:08.177901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09144686-2dcd-4117-8f87-23aab2797fe9 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:08.178206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09144686-2dcd-4117-8f87-23aab2797fe9 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:08.178469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09144686-2dcd-4117-8f87-23aab2797fe9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:08.187198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09144686-2dcd-4117-8f87-23aab2797fe9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:08.188839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09144686-2dcd-4117-8f87-23aab2797fe9 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:26:08.197100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1212/2405] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:26:08.477959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:08.501668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83177c-c341-45c7-88da-e8fb5bf568ad None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:08.511620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:08.512588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:08.513125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:08.513757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:08.517951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:08.532262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:08.534175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:26:08.547529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1194/2406] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:26:18.193807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:18.542512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=9 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:24.262478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:24.262943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:24.263137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:24.304674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:26:24.305957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:24.335821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1213/2407] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Aug 1 14:26:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:26:24.349242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:24.415684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f577d70-e300-435c-a5d7-c1983355dc1e service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:24.422394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8d9d0-ba88-4b68-a384-c044967f3de3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:24.423029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8d9d0-ba88-4b68-a384-c044967f3de3 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:24.423309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8d9d0-ba88-4b68-a384-c044967f3de3 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:24.423786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8d9d0-ba88-4b68-a384-c044967f3de3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:24.432504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8d9d0-ba88-4b68-a384-c044967f3de3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:24.434117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8d9d0-ba88-4b68-a384-c044967f3de3 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:26:24.443198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1195/2408] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:26:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:26:34.333703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=20 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:34.439899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=20 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:41.821061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:41.841336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03aff1b-1c27-42e0-b619-672886ea69a8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:41.845237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:41.845852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:41.846130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:41.846404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:41.848223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:41.860720 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:41.862157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:26:41.871749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1214/2409] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:26:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:26:46.560013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:46.560809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:46.560809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:46.588388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:26:46.590087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:46.601660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1196/2410] 10.222.0.17 () {58 vars in 967 bytes} [Mon Aug 1 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:26:46.616572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:46.629204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba8ad-a78e-446a-8db0-dd9758c085de service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:46.634726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:46.635419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:46.635741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:46.636019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:46.646671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:46.648344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:26:46.660948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1215/2411] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:26:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:26:55.352787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:55.375760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e11081-2dff-4b22-98e3-b5d5c84f1bfa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:55.379904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:55.380634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:55.381000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:55.381341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:55.383702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:55.516567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:55.517984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:26:55.541431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1197/2412] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:26:56.655495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=16 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:26:59.971361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:26:59.973049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:26:59.988741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26775eca-0907-4876-87ed-fa32fc20eed5 service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:26:59.993452 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:26:59.994167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:26:59.994443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:26:59.994712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] PATH_INFO: `/v3/users/c487e443eb274090bce842e855b015ec` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:26:59.995987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c487e443eb274090bce842e855b015ec)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:26:59.998159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.054306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c487e443eb274090bce842e855b015ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.054952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c487e443eb274090bce842e855b015ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.068359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c487e443eb274090bce842e855b015ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.076638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c487e443eb274090bce842e855b015ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.086215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bc76402e-1f38-4b33-9e00-4b09aa93398f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c487e443eb274090bce842e855b015ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.096103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1216/2413] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:26:59 2022] DELETE /identity/v3/users/c487e443eb274090bce842e855b015ec => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:00.289630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.301858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6f55c-e0c0-4f66-bedd-155856eba70a service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.376314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.377134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.377533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.377914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] PATH_INFO: `/v3/projects/7b6f2af7de6b4e888e702c78f655448a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.379581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6f2af7de6b4e888e702c78f655448a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:00.381653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.429298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6f2af7de6b4e888e702c78f655448a', 'request_id': 'req-48739d09-841a-4beb-bfda-b11c98b3bb72'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.431351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:00.437297 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6f2af7de6b4e888e702c78f655448a', 'request_id': 'req-48739d09-841a-4beb-bfda-b11c98b3bb72'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.445053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6f2af7de6b4e888e702c78f655448a', 'request_id': 'req-48739d09-841a-4beb-bfda-b11c98b3bb72'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.457880 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] The token cache is being invalidate because project 7b6f2af7de6b4e888e702c78f655448a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:00.458169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.458814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-48739d09-841a-4beb-bfda-b11c98b3bb72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:00.460381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1198/2414] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:00 2022] DELETE /identity/v3/projects/7b6f2af7de6b4e888e702c78f655448a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:00.465471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.465773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.466140 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.531074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:00.532343 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:00.595974 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1217/2415] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:00.605842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.606132 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.606432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.627328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:00.628602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:00.636759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1199/2416] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:00.642192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.656965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537899a7-92f3-4e37-a94f-a935c638a609 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.663799 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.664709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.665137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.665558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.666669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:00.667137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:00.670331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.684150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1218/2417] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:00.689579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.699280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542f377-06bd-45c2-aacd-d386631b6c93 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.703980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.704589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.704867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.705139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.705806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:00.707810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.749226 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1200/2418] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:00 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:00.754409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.765167 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438d8d7-9990-4767-b65e-2d92670ae69b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.791439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:00.827023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.827757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.828043 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.828320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.829001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:00.830931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.862843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1219/2419] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:00 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:00.864835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.881361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a0ca4-a78b-4c3f-bc15-cdce27efda63 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.888668 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.889509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.889965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.890425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.891337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:00.894267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.909902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1201/2420] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:00.917381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.939878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a536b-b3c4-4a8a-bc45-3a8d686df440 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.947134 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010d1f9-3618-40d6-b70f-0dc9307b54d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.948182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d1f9-3618-40d6-b70f-0dc9307b54d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.948728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d1f9-3618-40d6-b70f-0dc9307b54d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.949124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d1f9-3618-40d6-b70f-0dc9307b54d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:00.951290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010d1f9-3618-40d6-b70f-0dc9307b54d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:00.952398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010d1f9-3618-40d6-b70f-0dc9307b54d9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:00.966526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1220/2421] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:00.973714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:00.990875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a5fc0-02d7-4f33-9e85-af2fbd103f8a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:00.997026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:00.998897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:00.999253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:00.999643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] PATH_INFO: `/v3/projects/00a93cc41a184683a1fb82acfa2660a6/users/4079529de70f4994ab9f1f3dd5cb3fd7/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:01.030105 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a93cc41a184683a1fb82acfa2660a6, user_id=4079529de70f4994ab9f1f3dd5cb3fd7, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:01.035168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:01.056495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1202/2422] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:00 2022] PUT /identity/v3/projects/00a93cc41a184683a1fb82acfa2660a6/users/4079529de70f4994ab9f1f3dd5cb3fd7/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:01.062256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:01.062751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:01.062851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:01.090078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:01.092159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:01.164093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1221/2423] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:01.174935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:01.176751 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:01.250893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec796-9d8a-4b33-b298-79a6a355d717 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:01.254614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ce12d-69a9-467d-b327-f732aadeb4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:01.255242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ce12d-69a9-467d-b327-f732aadeb4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:01.255534 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ce12d-69a9-467d-b327-f732aadeb4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:01.255813 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ce12d-69a9-467d-b327-f732aadeb4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:01.266110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ce12d-69a9-467d-b327-f732aadeb4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:01.267655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ce12d-69a9-467d-b327-f732aadeb4cf service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:01.280204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1203/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:02.392966 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:02.404729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad5b01-dcd3-44cc-8196-17c07e5a1155 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:02.409162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:02.409880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:02.410365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:02.410853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:02.413738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:02.428380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:02.430544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:02.441927 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1222/2425] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:07.096888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:07.097519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:07.097519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:07.132236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:07.134299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:07.134436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] MFA Rules not processed for user `4079529de70f4994ab9f1f3dd5cb3fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:07.135904 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:07.150367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143aca2-7c61-44ba-9a36-50cc103b014e service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:07.156909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:07.157719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:07.158087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:07.158414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] PATH_INFO: `/v3/users/9eda4e19122444cf9cb2b6941adff9d5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:07.160386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eda4e19122444cf9cb2b6941adff9d5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:07.162856 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:07.198688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:07.245991 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1204/2426] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:07.255395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:07.257159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:07.257847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eda4e19122444cf9cb2b6941adff9d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.258710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eda4e19122444cf9cb2b6941adff9d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.280893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eda4e19122444cf9cb2b6941adff9d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.298457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eda4e19122444cf9cb2b6941adff9d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.313992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eda4e19122444cf9cb2b6941adff9d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.335835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1223/2427] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:07 2022] DELETE /identity/v3/users/9eda4e19122444cf9cb2b6941adff9d5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:07.437360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5a6dd-f15d-4032-bef5-97a9ddc584a0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:07.591182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:07.591874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:07.592159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:07.592437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:07.594542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:07.608857 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:07.613262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:07.635969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:07.706456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:07.708246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaa10d-de37-4aef-8655-0a78a70ee67d service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:07.720463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1205/2428] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:07.743490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c35d8d-d7d4-4fae-aa07-66655102716a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:07.749243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:07.749993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:07.750351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:07.750706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] PATH_INFO: `/v3/projects/3092982f3c6f4851a5a6e84bf7872d11` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:07.752124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3092982f3c6f4851a5a6e84bf7872d11)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:07.754293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:07.828895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3092982f3c6f4851a5a6e84bf7872d11', 'request_id': 'req-ead04fe3-933c-43a2-b44d-5165f79807b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.836194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3092982f3c6f4851a5a6e84bf7872d11', 'request_id': 'req-ead04fe3-933c-43a2-b44d-5165f79807b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.848327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3092982f3c6f4851a5a6e84bf7872d11', 'request_id': 'req-ead04fe3-933c-43a2-b44d-5165f79807b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.862866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] The token cache is being invalidate because project 3092982f3c6f4851a5a6e84bf7872d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:07.863335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.864520 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ead04fe3-933c-43a2-b44d-5165f79807b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:07.867316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1224/2429] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:07 2022] DELETE /identity/v3/projects/3092982f3c6f4851a5a6e84bf7872d11 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:07.873288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:07.875029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:07.875475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:07.902058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:07.903856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:07.929149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:07.980248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1206/2430] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:07.988082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:07.988384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:07.988670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.021360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:08.023141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:08.034275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1225/2431] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:08.044586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.056011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6193-2441-4d5a-8ad0-834157c3af13 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.062448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.063386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.064141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.064857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.065780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:08.066091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.067964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.084054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1207/2432] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:08.088772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.101359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31a1a1-1825-448f-9472-3e42421d4a4e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.107395 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.108206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.108495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.108876 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.109450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.112490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.151494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1226/2433] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:08 2022] POST /identity/v3/projects => generated 405 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:08.156264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.168380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d07f-b156-47a4-b967-b74358798796 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.242615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.243319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.243675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.243928 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.246275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.248811 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.277073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1208/2434] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:08 2022] POST /identity/v3/users => generated 303 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:08.281447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.295163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f327aff-4070-4f40-89cb-909c2a4b6a5e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.300899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.301552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.301842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.302458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.303199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.304909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.317850 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1227/2435] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:08.321969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.337884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c120c37-2515-4494-8a3b-a0ea74a28f4c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.343167 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a5ccc-f803-441f-964e-cd5e0baa9253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.344194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5ccc-f803-441f-964e-cd5e0baa9253 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.344543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5ccc-f803-441f-964e-cd5e0baa9253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.346056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5ccc-f803-441f-964e-cd5e0baa9253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.346795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a5ccc-f803-441f-964e-cd5e0baa9253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.348579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a5ccc-f803-441f-964e-cd5e0baa9253 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.361102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1209/2436] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:08.365602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.377321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d9fc2-07c2-448a-bce0-d1614a6503fb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.383499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.384166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.384454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.384736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] PATH_INFO: `/v3/projects/3525f63bddef4cc28822617341e99993/users/ed52d3ce24d74c69ab1dac9408acb0e2/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.402321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3525f63bddef4cc28822617341e99993, user_id=ed52d3ce24d74c69ab1dac9408acb0e2, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.405130 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.421155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1228/2437] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:08 2022] PUT /identity/v3/projects/3525f63bddef4cc28822617341e99993/users/ed52d3ce24d74c69ab1dac9408acb0e2/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:08.426690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9f2f4-0318-4de8-8793-ddc205ef0fb3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.427078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9f2f4-0318-4de8-8793-ddc205ef0fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.427498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9f2f4-0318-4de8-8793-ddc205ef0fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.447914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-70a9f2f4-0318-4de8-8793-ddc205ef0fb3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:08.449295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-70a9f2f4-0318-4de8-8793-ddc205ef0fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:08.527777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1210/2438] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:08.538572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:08.543390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:08.587629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:08.615920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166712e-bd3a-405a-96a7-6e941a19ccb0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:08.619716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29451958-8237-464f-a38f-739fadc053cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:08.620353 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451958-8237-464f-a38f-739fadc053cb service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:08.620637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451958-8237-464f-a38f-739fadc053cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:08.620911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451958-8237-464f-a38f-739fadc053cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:08.631193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29451958-8237-464f-a38f-739fadc053cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:08.634155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29451958-8237-464f-a38f-739fadc053cb service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:08.645736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1229/2439] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:09.279530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.281318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.281803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.303874 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] MFA Rules not processed for user `4079529de70f4994ab9f1f3dd5cb3fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:09.305468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:09.349425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:09.405123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1211/2440] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:09.461356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.461964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.461964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.499199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] MFA Rules not processed for user `ed52d3ce24d74c69ab1dac9408acb0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:09.501256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:09.592609 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1230/2441] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:09.597540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:09.609064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf6fd9-6da1-43ed-9620-98c5aa3d6220 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:09.614884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:09.615838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.616120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.616550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.617415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:09.619360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:09.670780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1212/2442] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:09 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:09.674865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:09.688841 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc8799-eb00-4980-8897-3d266eaf4eb0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:09.761723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:09.762424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.762973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.763409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.764279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:09.766241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:09.793383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1231/2443] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:09 2022] POST /identity/v3/users => generated 308 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:09.797694 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:09.810107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0dfc4-6c1b-4bce-80fb-7258377b37a6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:09.816191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:09.817122 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.817634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.818082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.818944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:09.820827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:09.834478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1213/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:09.839039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:09.850774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391bc2-c303-4466-84ca-e0475a6972cc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:09.857455 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:09.858377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.858981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.859256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] PATH_INFO: `/v3/projects/c47d975a296e490aa2ff17df5344cd26/users/5d2e7e1499e7468eb290bedb49f2c361/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.878963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47d975a296e490aa2ff17df5344cd26, user_id=5d2e7e1499e7468eb290bedb49f2c361, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:09.882808 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:09.901044 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1232/2445] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:09 2022] PUT /identity/v3/projects/c47d975a296e490aa2ff17df5344cd26/users/5d2e7e1499e7468eb290bedb49f2c361/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:09.905271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:09.924010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb1129-6c15-4aca-9cbd-19264e0fcb35 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:09.993747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f57fa1-b368-442d-9524-719d6bc532cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:09.994681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57fa1-b368-442d-9524-719d6bc532cc admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:09.994971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57fa1-b368-442d-9524-719d6bc532cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:09.995302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57fa1-b368-442d-9524-719d6bc532cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:09.996217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f57fa1-b368-442d-9524-719d6bc532cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:09.998653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f57fa1-b368-442d-9524-719d6bc532cc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:10.010595 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1214/2446] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:10.015688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:10.027642 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93222d9f-738f-4dcf-bd2a-94ad45784028 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:10.044798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:10.045505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:10.045819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:10.046099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] PATH_INFO: `/v3/projects/c47d975a296e490aa2ff17df5344cd26/users/5d2e7e1499e7468eb290bedb49f2c361/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:10.048501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47d975a296e490aa2ff17df5344cd26, user_id=5d2e7e1499e7468eb290bedb49f2c361, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:10.052006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:10.065880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1233/2447] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:10 2022] PUT /identity/v3/projects/c47d975a296e490aa2ff17df5344cd26/users/5d2e7e1499e7468eb290bedb49f2c361/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:10.941859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:10.942422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:10.942523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:10.983697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] MFA Rules not processed for user `5d2e7e1499e7468eb290bedb49f2c361`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:10.985249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:11.066030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1215/2448] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:11.729056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:11.731426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:11.782391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:11.878950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd6ad-6840-4435-a66c-292fcd8f64cf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:11.886126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:11.887339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:11.888955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:11.889508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:11.892208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:11.908820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:11.910787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:11.938020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1234/2449] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:12.558976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:12.600593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:12.645263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a68b5-4fc1-4249-ad8e-7ba1f55d8367 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:12.650385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:12.651054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:12.651339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:12.651675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] PATH_INFO: `/v3/users/ed52d3ce24d74c69ab1dac9408acb0e2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:12.653720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed52d3ce24d74c69ab1dac9408acb0e2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:12.655780 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:12.731187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed52d3ce24d74c69ab1dac9408acb0e2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:12.732218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed52d3ce24d74c69ab1dac9408acb0e2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:12.746923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed52d3ce24d74c69ab1dac9408acb0e2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:12.755767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed52d3ce24d74c69ab1dac9408acb0e2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:12.756697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed52d3ce24d74c69ab1dac9408acb0e2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:12.773313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1216/2450] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:12 2022] DELETE /identity/v3/users/ed52d3ce24d74c69ab1dac9408acb0e2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:13.075995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:13.090454 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ac2a-a571-4128-bd08-a81c2b0816b4 service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:13.160141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:13.160932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:13.161219 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:13.161493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] PATH_INFO: `/v3/projects/3525f63bddef4cc28822617341e99993` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:13.162668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3525f63bddef4cc28822617341e99993)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:13.164607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:13.217357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3525f63bddef4cc28822617341e99993', 'request_id': 'req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.223273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3525f63bddef4cc28822617341e99993', 'request_id': 'req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.232351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3525f63bddef4cc28822617341e99993', 'request_id': 'req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.244162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] The token cache is being invalidate because project 3525f63bddef4cc28822617341e99993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:13.244447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.245091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.247213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1235/2451] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:13 2022] DELETE /identity/v3/projects/3525f63bddef4cc28822617341e99993 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:13.251961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:13.253000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:13.267980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d0f76-e493-4c57-bc27-c99bb9cc5dc7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:13.330617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:13.331249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:13.331572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:13.331850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] PATH_INFO: `/v3/users/5d2e7e1499e7468eb290bedb49f2c361` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:13.333173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2e7e1499e7468eb290bedb49f2c361)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:13.335196 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:13.385401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:13.389323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2e7e1499e7468eb290bedb49f2c361'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.390681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2e7e1499e7468eb290bedb49f2c361'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.404632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2e7e1499e7468eb290bedb49f2c361'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.414507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2e7e1499e7468eb290bedb49f2c361'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.417323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b50bd418-45ea-4cec-b479-cd966842c6d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2e7e1499e7468eb290bedb49f2c361'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.435840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1217/2452] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:13 2022] DELETE /identity/v3/users/5d2e7e1499e7468eb290bedb49f2c361 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:13.649075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:13.650691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:13.678272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b5627-f21d-4c8a-8702-e8e3e20bb49d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:13.716812 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:13.743176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:13.743849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:13.744197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:13.745135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] PATH_INFO: `/v3/projects/c47d975a296e490aa2ff17df5344cd26` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:13.746423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47d975a296e490aa2ff17df5344cd26)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:13.749122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:13.799997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47d975a296e490aa2ff17df5344cd26', 'request_id': 'req-2c607323-f9f2-4223-89e4-b0a853d9a6fa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.805819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47d975a296e490aa2ff17df5344cd26', 'request_id': 'req-2c607323-f9f2-4223-89e4-b0a853d9a6fa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.814030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47d975a296e490aa2ff17df5344cd26', 'request_id': 'req-2c607323-f9f2-4223-89e4-b0a853d9a6fa'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.825800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] The token cache is being invalidate because project c47d975a296e490aa2ff17df5344cd26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:13.826088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.826721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2c607323-f9f2-4223-89e4-b0a853d9a6fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:13.828275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1236/2453] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:13 2022] DELETE /identity/v3/projects/c47d975a296e490aa2ff17df5344cd26 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:13.833148 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:13.833615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:13.833710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:13.858055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:13.859295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:13.921055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1218/2454] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:13.929342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:13.929342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:13.929342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:13.950004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:13.951241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:13.959536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1237/2455] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:13.963817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:13.974631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9664ccd-b392-44b0-bec7-e885546fc270 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:13.979609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:13.980237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:13.980522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:13.980801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:13.982426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:13.982735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:13.984372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:13.997731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1219/2456] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:14.002232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.012495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789bcb8-8a93-4f03-be84-8b9e3b0cb614 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.017604 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.018602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.018814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.018907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.019748 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.021782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.051879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1238/2457] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:14 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:14.056208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.066642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b72f3-463d-4891-801f-f7b77d5a9841 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.132586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.133221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.133504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.133782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.134472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.136475 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.159705 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1220/2458] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:14 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:14.163796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.175272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09683fea-9ad4-4566-8a60-9ec0b5dece29 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.180641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.181298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.181585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.181867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.182586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.184314 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.196513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1239/2459] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:14.201460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.212181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5f741-b188-4182-9e3d-3fb303d0a82d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.217636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f9e61-8d3d-481a-a52e-7a5f72565df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.218312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f9e61-8d3d-481a-a52e-7a5f72565df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.218863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f9e61-8d3d-481a-a52e-7a5f72565df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.219143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f9e61-8d3d-481a-a52e-7a5f72565df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.219901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f9e61-8d3d-481a-a52e-7a5f72565df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.221696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f9e61-8d3d-481a-a52e-7a5f72565df9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.233516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1221/2460] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:14.237861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.248871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c34b0-0709-4416-adc4-b639b1fe7ec4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.254041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.254761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.255050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.255329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] PATH_INFO: `/v3/projects/f0fc5d0b21c045809b7fabacb0d65c0f/users/c2330a42720d4ee4abd970fb82e3d038/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.272560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0fc5d0b21c045809b7fabacb0d65c0f, user_id=c2330a42720d4ee4abd970fb82e3d038, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.275684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.292798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1240/2461] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:14 2022] PUT /identity/v3/projects/f0fc5d0b21c045809b7fabacb0d65c0f/users/c2330a42720d4ee4abd970fb82e3d038/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:14.303576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.303872 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.304150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.334849 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] MFA Rules not processed for user `c2330a42720d4ee4abd970fb82e3d038`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:14.336143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:14.354833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:14.408419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1222/2462] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:14.413057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.480813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5ac9d-e95c-4ad5-80a5-1b0facc4f809 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.485560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.486260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.487005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.487005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.487722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.489684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.502738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:14.526633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1241/2463] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:14 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:14.531056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.542768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb2cea-92d1-42a0-8ea7-68b117b0412f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.603178 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.603829 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.604110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.604381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.605043 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.607002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.632371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1223/2464] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:14 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:14.633564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.645396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac3a43-261c-4b75-a8ca-bf4bf1edd29b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.650233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.650887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.651230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.651475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.652200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.653851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.665260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1242/2465] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:14.669682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.679912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25b7cf-142c-4bda-af01-282660a4954e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.685306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.685967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.686255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.686548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] PATH_INFO: `/v3/projects/ae6be59580f94f7f8fda3973000477bf/users/f58c5dd99ab94178bd216b3d98de8073/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.702552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6be59580f94f7f8fda3973000477bf, user_id=f58c5dd99ab94178bd216b3d98de8073, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.705295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.718185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1224/2466] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:14 2022] PUT /identity/v3/projects/ae6be59580f94f7f8fda3973000477bf/users/f58c5dd99ab94178bd216b3d98de8073/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:14.723651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.732826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39336ce3-58c7-47d2-b88f-067ffbc9d6e7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.788056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c2f64-59dd-4552-960f-d6bb77c53f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.788772 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c2f64-59dd-4552-960f-d6bb77c53f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.788957 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c2f64-59dd-4552-960f-d6bb77c53f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.789136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c2f64-59dd-4552-960f-d6bb77c53f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.789826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c2f64-59dd-4552-960f-d6bb77c53f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.791410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c2f64-59dd-4552-960f-d6bb77c53f52 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.801280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1243/2467] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:14.805630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.816759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d62cc-27ff-44e5-ad82-db798cf32968 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:14.821361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:14.822013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.822296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.822581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] PATH_INFO: `/v3/projects/ae6be59580f94f7f8fda3973000477bf/users/f58c5dd99ab94178bd216b3d98de8073/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.824576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6be59580f94f7f8fda3973000477bf, user_id=f58c5dd99ab94178bd216b3d98de8073, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:14.827957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:14.839654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1225/2468] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:14 2022] PUT /identity/v3/projects/ae6be59580f94f7f8fda3973000477bf/users/f58c5dd99ab94178bd216b3d98de8073/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:14.851505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:14.852190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:14.852190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:14.880795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] MFA Rules not processed for user `f58c5dd99ab94178bd216b3d98de8073`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:14.882075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:14.953396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1244/2469] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:14 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:14.964603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:14.966452 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:15.040060 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f27d4-7ada-4261-bd7f-ef233b821105 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:15.044239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098e2d3-c9d2-41ef-a044-3cdca0a06c6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:15.044832 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098e2d3-c9d2-41ef-a044-3cdca0a06c6a service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:15.045114 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098e2d3-c9d2-41ef-a044-3cdca0a06c6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:15.045386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098e2d3-c9d2-41ef-a044-3cdca0a06c6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:15.054373 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098e2d3-c9d2-41ef-a044-3cdca0a06c6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:15.055844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098e2d3-c9d2-41ef-a044-3cdca0a06c6a service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:15.066401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1226/2470] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:14 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:15.493896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:15.557551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db7a0a-1ce7-4483-976a-1fbdb60e571e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:15.562017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:15.562739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:15.563027 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:15.563303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] PATH_INFO: `/v3/users/c2330a42720d4ee4abd970fb82e3d038` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:15.564531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2330a42720d4ee4abd970fb82e3d038)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:15.566464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:15.568146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:15.622527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2330a42720d4ee4abd970fb82e3d038'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:15.623392 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2330a42720d4ee4abd970fb82e3d038'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:15.635686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2330a42720d4ee4abd970fb82e3d038'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:15.647416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2330a42720d4ee4abd970fb82e3d038'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:15.655363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2330a42720d4ee4abd970fb82e3d038'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:15.667199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1245/2471] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:15 2022] DELETE /identity/v3/users/c2330a42720d4ee4abd970fb82e3d038 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:15.669390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:15.669804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:15.670112 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:15.678288 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:15.692690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:15.694096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:15.759471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1227/2472] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:15.769825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:15.771377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:15.847263 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a6a1b-6f9f-404d-b817-4e3807f48d7b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:15.852146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:15.852965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:15.853439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:15.853641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:15.864171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:15.865614 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:15.875514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1246/2473] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:16.159975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:16.161503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:16.175655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf06a2-73a8-43fe-a16f-7d7115956e20 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:16.195817 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:16.196640 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:16.197096 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:16.197329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] PATH_INFO: `/v3/projects/f0fc5d0b21c045809b7fabacb0d65c0f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:16.198642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0fc5d0b21c045809b7fabacb0d65c0f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:16.200824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:16.248399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0fc5d0b21c045809b7fabacb0d65c0f', 'request_id': 'req-9454398d-d2c8-46a7-b519-92954f2a6dd4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.253913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0fc5d0b21c045809b7fabacb0d65c0f', 'request_id': 'req-9454398d-d2c8-46a7-b519-92954f2a6dd4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.261428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0fc5d0b21c045809b7fabacb0d65c0f', 'request_id': 'req-9454398d-d2c8-46a7-b519-92954f2a6dd4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.273314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] The token cache is being invalidate because project f0fc5d0b21c045809b7fabacb0d65c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:16.273606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.274245 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.275877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1228/2474] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:16 2022] DELETE /identity/v3/projects/f0fc5d0b21c045809b7fabacb0d65c0f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:16.280751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:16.295246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc674d4-b53f-4289-bde2-635cbe446e19 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:16.357691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:16.358602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:16.358886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:16.359162 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] PATH_INFO: `/v3/users/f58c5dd99ab94178bd216b3d98de8073` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:16.360382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58c5dd99ab94178bd216b3d98de8073)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:16.362305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:16.385851 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:16.411977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58c5dd99ab94178bd216b3d98de8073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.412627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58c5dd99ab94178bd216b3d98de8073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.425137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58c5dd99ab94178bd216b3d98de8073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.432939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58c5dd99ab94178bd216b3d98de8073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.441217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d8dad8ab-0b08-4657-8540-5459abb3b498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58c5dd99ab94178bd216b3d98de8073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.447289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1247/2475] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:16 2022] DELETE /identity/v3/users/f58c5dd99ab94178bd216b3d98de8073 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:16.745842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:16.748091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:16.761895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454398d-d2c8-46a7-b519-92954f2a6dd4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:16.820256 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:16.820946 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:16.821211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:16.821482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] PATH_INFO: `/v3/projects/ae6be59580f94f7f8fda3973000477bf` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:16.822644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6be59580f94f7f8fda3973000477bf)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:16.824638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:16.857000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:16.873004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6be59580f94f7f8fda3973000477bf', 'request_id': 'req-54e812f5-4536-4583-a112-cd92e1514754'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.878378 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6be59580f94f7f8fda3973000477bf', 'request_id': 'req-54e812f5-4536-4583-a112-cd92e1514754'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.885615 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6be59580f94f7f8fda3973000477bf', 'request_id': 'req-54e812f5-4536-4583-a112-cd92e1514754'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.897568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] The token cache is being invalidate because project ae6be59580f94f7f8fda3973000477bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:16.897878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.898514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-54e812f5-4536-4583-a112-cd92e1514754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:16.900364 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1229/2476] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:16 2022] DELETE /identity/v3/projects/ae6be59580f94f7f8fda3973000477bf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:16.905997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:16.906521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:16.906620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:16.926670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:16.927929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:16.988006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1248/2477] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:16.995773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:16.996062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:16.996332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.016401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:17.017606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:17.025661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1230/2478] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:17.030433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.041986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b403b-f468-4b04-8ca1-e4fbb58ff5a2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.047080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.047755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.048034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.048311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.049006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:17.049319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.050923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.063417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1249/2479] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:17.067809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.077290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb726b1-8eef-4768-94ee-449e9eee334b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.081897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.082510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.082790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.083062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.083756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.085647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.116044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1231/2480] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:17 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:17.120215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.130736 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dd71-3953-40f1-912f-a6667c7e3ed9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.190764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.191441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.191762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.192069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.192713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.194825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.227241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1250/2481] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:17 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:17.228544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.239641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8932287-d976-4c90-b49d-7fe14997e99b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.244380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.244999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.245282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.245555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.246265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.247909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.258518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1232/2482] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:17.263132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.273803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb33a68-c21d-44fd-8261-0738fd2de7e9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.279591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabd18e-e025-4019-b60a-8dcb061f582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.280413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabd18e-e025-4019-b60a-8dcb061f582a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.280690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabd18e-e025-4019-b60a-8dcb061f582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.280946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabd18e-e025-4019-b60a-8dcb061f582a admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.281666 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabd18e-e025-4019-b60a-8dcb061f582a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.283275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabd18e-e025-4019-b60a-8dcb061f582a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.294800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1251/2483] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:17.299102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.309593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b8377-6931-484e-b611-c82adfacba92 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.314330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.315006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.315290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.315601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] PATH_INFO: `/v3/projects/efd0ecf7b0314e3085d8b2ebc743c183/users/9108bd11e056438eb29bd63babd8a4d9/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.335955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd0ecf7b0314e3085d8b2ebc743c183, user_id=9108bd11e056438eb29bd63babd8a4d9, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.338718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.356643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1233/2484] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:17 2022] PUT /identity/v3/projects/efd0ecf7b0314e3085d8b2ebc743c183/users/9108bd11e056438eb29bd63babd8a4d9/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:17.368246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.368467 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.368740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.381948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:17.398916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] MFA Rules not processed for user `9108bd11e056438eb29bd63babd8a4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:17.400143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:17.463934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1252/2485] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:17.475792 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.549714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66815bd6-c87b-42d9-9f4f-b20a4e638bb7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.554463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.555079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.555360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.555657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.556390 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.558286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.586387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1234/2486] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:17 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:17.590846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.601860 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98439315-86cd-4854-980b-9b3c64c33de2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.658204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.658759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.659046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.659325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.660081 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.662035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.684385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1253/2487] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:17 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:17.688407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.698684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4f22e-2070-499f-b538-d43d47378ab0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.703594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.704205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.704482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.704756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.705447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.707037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.718234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1235/2488] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:17.722612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.733229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8fee-1397-4933-b2c6-a2ab17ea1e41 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.737840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.738495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.738773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.739048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] PATH_INFO: `/v3/projects/17459b4a328144d3a158e1c053bb76e3/users/fc7cf1004db148e896133a24586643dc/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.753450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17459b4a328144d3a158e1c053bb76e3, user_id=fc7cf1004db148e896133a24586643dc, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.756124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.769129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1254/2489] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:17 2022] PUT /identity/v3/projects/17459b4a328144d3a158e1c053bb76e3/users/fc7cf1004db148e896133a24586643dc/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:17.773113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.784128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba567f7-1846-4183-b631-1a369ae1df0b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.808221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:17.844068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556498a7-f0bb-407a-8935-33f119f49ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.844683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556498a7-f0bb-407a-8935-33f119f49ada admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.844972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556498a7-f0bb-407a-8935-33f119f49ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.845251 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556498a7-f0bb-407a-8935-33f119f49ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.845943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556498a7-f0bb-407a-8935-33f119f49ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.847614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556498a7-f0bb-407a-8935-33f119f49ada admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.857908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1236/2490] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:17.862132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:17.872347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783c289-1da7-44f8-966b-0bc95b7e1c61 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:17.876939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:17.877597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.877880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.878158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] PATH_INFO: `/v3/projects/17459b4a328144d3a158e1c053bb76e3/users/fc7cf1004db148e896133a24586643dc/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.880259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17459b4a328144d3a158e1c053bb76e3, user_id=fc7cf1004db148e896133a24586643dc, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:17.882900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:17.893171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1255/2491] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:17 2022] PUT /identity/v3/projects/17459b4a328144d3a158e1c053bb76e3/users/fc7cf1004db148e896133a24586643dc/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:17.903845 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:17.904138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:17.904419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:17.932360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] MFA Rules not processed for user `fc7cf1004db148e896133a24586643dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:17.933510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:18.001804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1237/2492] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:18.013636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:18.015398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:18.070948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:18.074725 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad8f21-82ef-4b29-90a0-3f4bd3a35b0d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:18.079249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:18.080228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:18.080511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:18.080804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:18.091447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:18.092907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:18.103323 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1256/2493] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:18 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:18.181391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:18.192859 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cccde6-1cfe-410f-a683-b20065ed4697 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:18.196518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ae90f-8118-49cc-bc0f-ab0ab270d679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:18.197267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae90f-8118-49cc-bc0f-ab0ab270d679 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:18.197665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae90f-8118-49cc-bc0f-ab0ab270d679 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:18.199833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae90f-8118-49cc-bc0f-ab0ab270d679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:18.258802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ae90f-8118-49cc-bc0f-ab0ab270d679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:18.260296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ae90f-8118-49cc-bc0f-ab0ab270d679 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:18.270274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1238/2494] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:21.129430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:21.219862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f25c6-b9ea-4b00-896e-a5ea00850952 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:21.226067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:21.226801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:21.227150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:21.227538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] PATH_INFO: `/v3/users/9108bd11e056438eb29bd63babd8a4d9` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:21.228790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9108bd11e056438eb29bd63babd8a4d9)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:21.231425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:21.298269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9108bd11e056438eb29bd63babd8a4d9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:21.299005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9108bd11e056438eb29bd63babd8a4d9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:21.314491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9108bd11e056438eb29bd63babd8a4d9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:21.324999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9108bd11e056438eb29bd63babd8a4d9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:21.336319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9108bd11e056438eb29bd63babd8a4d9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:21.345577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1257/2495] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:21 2022] DELETE /identity/v3/users/9108bd11e056438eb29bd63babd8a4d9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:21.351543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:21.352721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:21.353445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:21.377067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:21.378474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:21.427055 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:21.447091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1239/2496] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:21.459073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:21.460564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:21.532651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832805b4-4ed8-41ee-84fa-ca3363a004cd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:21.536786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:21.537431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:21.537709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:21.537980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:21.548945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:21.550404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:21.561602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1258/2497] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:21.875938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:21.877768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:21.907435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41636-981d-44c0-ba8a-dd7bff129a9c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:21.915095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:21.915916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:21.916070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:21.916351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] PATH_INFO: `/v3/projects/efd0ecf7b0314e3085d8b2ebc743c183` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:21.917540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd0ecf7b0314e3085d8b2ebc743c183)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:21.919559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:22.003482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd0ecf7b0314e3085d8b2ebc743c183', 'request_id': 'req-b4a83720-f684-4fa0-bb97-258b832b58f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.009905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd0ecf7b0314e3085d8b2ebc743c183', 'request_id': 'req-b4a83720-f684-4fa0-bb97-258b832b58f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.018616 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd0ecf7b0314e3085d8b2ebc743c183', 'request_id': 'req-b4a83720-f684-4fa0-bb97-258b832b58f3'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.031404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] The token cache is being invalidate because project efd0ecf7b0314e3085d8b2ebc743c183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:22.031819 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.032485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.034009 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1240/2498] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:21 2022] DELETE /identity/v3/projects/efd0ecf7b0314e3085d8b2ebc743c183 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:22.037940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:22.048372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c4871-f69a-4dd4-b701-86f984ff9694 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:22.057581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:22.111045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:22.111734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:22.112020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:22.112299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] PATH_INFO: `/v3/users/fc7cf1004db148e896133a24586643dc` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:22.113488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7cf1004db148e896133a24586643dc)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:22.116024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:22.170495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7cf1004db148e896133a24586643dc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.171623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7cf1004db148e896133a24586643dc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.185845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7cf1004db148e896133a24586643dc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.198070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7cf1004db148e896133a24586643dc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.209249 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7940eeea-a961-42e3-bdc3-9a15b2e3cc30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7cf1004db148e896133a24586643dc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.216559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1259/2499] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:22 2022] DELETE /identity/v3/users/fc7cf1004db148e896133a24586643dc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:22.552501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:22.554003 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:22.575326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a83720-f684-4fa0-bb97-258b832b58f3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:22.650461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:22.651243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:22.651803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:22.652200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] PATH_INFO: `/v3/projects/17459b4a328144d3a158e1c053bb76e3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:22.654180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17459b4a328144d3a158e1c053bb76e3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:22.656171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:22.715008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17459b4a328144d3a158e1c053bb76e3', 'request_id': 'req-21b6c599-b12d-438a-bc04-121a28522707'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.720878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17459b4a328144d3a158e1c053bb76e3', 'request_id': 'req-21b6c599-b12d-438a-bc04-121a28522707'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.728658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17459b4a328144d3a158e1c053bb76e3', 'request_id': 'req-21b6c599-b12d-438a-bc04-121a28522707'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.739816 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:22.744547 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] The token cache is being invalidate because project 17459b4a328144d3a158e1c053bb76e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:22.744942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.747762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-21b6c599-b12d-438a-bc04-121a28522707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:22.750146 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1241/2500] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:22 2022] DELETE /identity/v3/projects/17459b4a328144d3a158e1c053bb76e3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:22.756285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:22.756577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:22.756854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:22.777212 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:22.778970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:22.840900 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1260/2501] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:22.848856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:22.849149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:22.849427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:22.871436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:22.872917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:22.881715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1242/2502] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:22.885960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:22.896066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23b6f1-21de-4d4f-ae29-dca113db976d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:22.901058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:22.901674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:22.901961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:22.902244 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:22.903525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:22.903837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:22.905503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:22.917806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1261/2503] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:22.922903 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:22.932278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a47d3a-9d37-4752-9a2c-ff8a0ac8909f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:22.938464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:22.939090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:22.939375 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:22.939679 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:22.940354 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:22.942257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:22.972180 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1243/2504] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:22 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:22.976450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:22.987430 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093b6e7-cbf5-40e9-b61a-adafca9fe096 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.005939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:23.051578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.052229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.052508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.052788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.053483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.055446 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.081298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1262/2505] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:22 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:23.085514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.097286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb9950-fe01-4d56-bc23-fa5cb3562422 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.103264 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.103938 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.104247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.104498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.105211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.107405 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.120886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1244/2506] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:23.122255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.132582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8e1b8-5558-4041-a310-00f76135be50 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.137324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f24d4-e5d4-4713-bb04-44c27f109a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.137940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f24d4-e5d4-4713-bb04-44c27f109a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.138225 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f24d4-e5d4-4713-bb04-44c27f109a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.138501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f24d4-e5d4-4713-bb04-44c27f109a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.139223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f24d4-e5d4-4713-bb04-44c27f109a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.140854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f24d4-e5d4-4713-bb04-44c27f109a40 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.155921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1263/2507] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:23.157337 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.168768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5d3df-7174-4810-a9d1-b187df8dbfd0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.173649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.174426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.174777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.175123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] PATH_INFO: `/v3/projects/9fc564b74a5d42ad8f3aec1003240f75/users/14e0f94a2f2c4509896a372576e9bc1e/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.193625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc564b74a5d42ad8f3aec1003240f75, user_id=14e0f94a2f2c4509896a372576e9bc1e, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.196037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.211046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1245/2508] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:23 2022] PUT /identity/v3/projects/9fc564b74a5d42ad8f3aec1003240f75/users/14e0f94a2f2c4509896a372576e9bc1e/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:23.223109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.223683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.223683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.256640 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] MFA Rules not processed for user `14e0f94a2f2c4509896a372576e9bc1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:23.258357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:23.342494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1264/2509] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:23.348985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.437025 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb182-6deb-46bf-b41a-e5eca1c728ce admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.442274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.442977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.443262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.443667 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.444406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.446628 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.487513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1246/2510] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:23 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:23.491565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.530976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342e1e-7873-43a0-9b6d-01606e5d60b2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.601487 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.602291 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.602679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.603057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.603898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.606528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.634438 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1265/2511] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:23 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:23.635660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.649152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec57a8-4756-4d65-9e22-34f3ec8186aa admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.661516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.662371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.662760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.663289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.664187 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.666423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.683533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1247/2512] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:23.687750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.702059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f6821-7286-4981-959b-beb2876162b2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.707777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.708558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.709760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.709760 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] PATH_INFO: `/v3/projects/0e364db9c20e41aebc645a64f5a0432d/users/d93239af88fb4ca3a04668d325684c60/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.728457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e364db9c20e41aebc645a64f5a0432d, user_id=d93239af88fb4ca3a04668d325684c60, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.732592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.739999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:23.765163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1266/2513] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:23 2022] PUT /identity/v3/projects/0e364db9c20e41aebc645a64f5a0432d/users/d93239af88fb4ca3a04668d325684c60/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:23.769174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.781918 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e7961-16de-4a87-b641-2a6a0829fdbb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.850115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a3796-bd4b-408f-914d-54bf0e695ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.850785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3796-bd4b-408f-914d-54bf0e695ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.851050 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3796-bd4b-408f-914d-54bf0e695ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.851330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3796-bd4b-408f-914d-54bf0e695ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.852092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a3796-bd4b-408f-914d-54bf0e695ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.853818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a3796-bd4b-408f-914d-54bf0e695ef9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.868901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1248/2514] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:23.870090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:23.882949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ace9e-dcc6-4b83-97ff-1343827a0fbd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:23.890296 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:23.891204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.891527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.891815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] PATH_INFO: `/v3/projects/0e364db9c20e41aebc645a64f5a0432d/users/d93239af88fb4ca3a04668d325684c60/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.894659 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e364db9c20e41aebc645a64f5a0432d, user_id=d93239af88fb4ca3a04668d325684c60, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:23.897481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:23.915592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1267/2515] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:23 2022] PUT /identity/v3/projects/0e364db9c20e41aebc645a64f5a0432d/users/d93239af88fb4ca3a04668d325684c60/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:23.923186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:23.923408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:23.923687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:23.938798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:23.956310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] MFA Rules not processed for user `d93239af88fb4ca3a04668d325684c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:23.957543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:24.053596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1249/2516] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:24.066411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:24.068253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:24.136495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037835-a739-4193-b4b0-4d5e0eba43f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:24.140199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9fb9a-2da7-44ea-953f-8d791f72b29a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:24.140818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9fb9a-2da7-44ea-953f-8d791f72b29a service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:24.141099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9fb9a-2da7-44ea-953f-8d791f72b29a service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:24.141373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9fb9a-2da7-44ea-953f-8d791f72b29a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:24.150679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9fb9a-2da7-44ea-953f-8d791f72b29a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:24.152151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9fb9a-2da7-44ea-953f-8d791f72b29a service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:24.162080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1268/2517] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:25.760782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:25.853046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec2279-0a65-4ae1-be46-82a486dc3c1f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:25.858120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:25.858767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:25.859079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:25.859353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] PATH_INFO: `/v3/users/14e0f94a2f2c4509896a372576e9bc1e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:25.860541 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e0f94a2f2c4509896a372576e9bc1e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:25.862642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:25.909032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e0f94a2f2c4509896a372576e9bc1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:25.909744 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e0f94a2f2c4509896a372576e9bc1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:25.922230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e0f94a2f2c4509896a372576e9bc1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:25.928990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e0f94a2f2c4509896a372576e9bc1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:25.929614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e0f94a2f2c4509896a372576e9bc1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:25.942555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1250/2518] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:25 2022] DELETE /identity/v3/users/14e0f94a2f2c4509896a372576e9bc1e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:25.947122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:25.947628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:25.947724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:25.966683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:25.967918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:26.037475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1269/2519] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:26.047798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:26.049432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:26.063785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:26.125358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db759dbe-bd55-494d-af72-81e137b4e964 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:26.129604 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:26.130311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:26.130708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:26.131027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:26.142814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:26.144290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:26.155590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1251/2520] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:26.532647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:26.534607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:26.550994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752180b-3aa2-4735-8dea-39252b1d58bf None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:26.555801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:26.556704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:26.557155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:26.557435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] PATH_INFO: `/v3/projects/9fc564b74a5d42ad8f3aec1003240f75` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:26.559151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc564b74a5d42ad8f3aec1003240f75)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:26.562393 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:26.620541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc564b74a5d42ad8f3aec1003240f75', 'request_id': 'req-6cdd5709-1378-4350-9248-92731aa33121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.629328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc564b74a5d42ad8f3aec1003240f75', 'request_id': 'req-6cdd5709-1378-4350-9248-92731aa33121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.637877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc564b74a5d42ad8f3aec1003240f75', 'request_id': 'req-6cdd5709-1378-4350-9248-92731aa33121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.650057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] The token cache is being invalidate because project 9fc564b74a5d42ad8f3aec1003240f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:26.650344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.651068 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.652928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1270/2521] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:26 2022] DELETE /identity/v3/projects/9fc564b74a5d42ad8f3aec1003240f75 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:26.656726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:26.669967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b7151-3ae9-4ec8-b70e-28cd9a14d02c service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:26.740458 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:26.741580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:26.741580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:26.742572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] PATH_INFO: `/v3/users/d93239af88fb4ca3a04668d325684c60` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:26.744406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93239af88fb4ca3a04668d325684c60)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:26.749366 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:26.800722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93239af88fb4ca3a04668d325684c60'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.801454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93239af88fb4ca3a04668d325684c60'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.816336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93239af88fb4ca3a04668d325684c60'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.824097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93239af88fb4ca3a04668d325684c60'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.825400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-171740f3-79bf-485e-93e1-34c00ced842d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93239af88fb4ca3a04668d325684c60'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:26.838771 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1252/2522] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:26 2022] DELETE /identity/v3/users/d93239af88fb4ca3a04668d325684c60 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:27.157879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.159610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:27.182146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd5709-1378-4350-9248-92731aa33121 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.191260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:27.265685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.266357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.266644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.266921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] PATH_INFO: `/v3/projects/0e364db9c20e41aebc645a64f5a0432d` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.268397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e364db9c20e41aebc645a64f5a0432d)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.270654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.338790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e364db9c20e41aebc645a64f5a0432d', 'request_id': 'req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:27.348753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e364db9c20e41aebc645a64f5a0432d', 'request_id': 'req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:27.359807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e364db9c20e41aebc645a64f5a0432d', 'request_id': 'req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:27.372545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] The token cache is being invalidate because project 0e364db9c20e41aebc645a64f5a0432d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:27.373012 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:27.373688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80016b68-b11a-4cd1-9e0a-ab65224a9a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:27.375540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1271/2523] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:27 2022] DELETE /identity/v3/projects/0e364db9c20e41aebc645a64f5a0432d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:27.380212 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.380800 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.381030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.401978 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:27.403597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:27.432882 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:27.470550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1253/2524] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:27.478699 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.478988 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.479272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.501045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:27.502305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:27.513810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1272/2525] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:27.521257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.531477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43998d2-7f5c-47e7-a78f-9396e2ce1ebb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.537085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.537888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.538210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.538558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.539343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:27.539748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.541639 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1896d-92b7-4150-a846-965b58395a0e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.563971 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1254/2526] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:27:27.564531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.575762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b715b-b5e9-43c7-b686-6ef5963e2791 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.581440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.582175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.582531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.582914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.583702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.585990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.587326 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.587922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.589026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.615942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1273/2527] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:27 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:27.619961 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] MFA Rules not processed for user `8631c9e2a3934738999f0315215ba682`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:27.620822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.621923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:27.698201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be00e0-174d-4295-b22f-0cd5ad793aa7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.703594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.704335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.704724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.704990 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.705508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.709449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.710348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1255/2528] 10.222.0.17 () {58 vars in 967 bytes} [Mon Aug 1 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:27.727272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:27.735087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1274/2529] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:27 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:27.737339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.738988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:27.740447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.757037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf4066-f814-4065-9960-a55c87a0d4bb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.762564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.763194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.763498 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.763793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.764512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.766342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.780890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1275/2530] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:27.785005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.795579 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3d47a-48f2-410e-86d9-f4b2aa60fec8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.800578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.801192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.801475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.801752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.802480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.804113 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.814867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1276/2531] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:27.818686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587402b-3102-48f6-b481-811b38ae2400 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.819171 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:27.825380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aee4c-0d06-4709-b8b8-dbb15ce6c503 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3690b0c864cb48a4b55774bb8875b644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.826242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aee4c-0d06-4709-b8b8-dbb15ce6c503 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.827254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aee4c-0d06-4709-b8b8-dbb15ce6c503 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.827686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aee4c-0d06-4709-b8b8-dbb15ce6c503 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.829503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de215286-196a-4df7-b449-e64c151fcecd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:27.834300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:27.834948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.835229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.835537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] PATH_INFO: `/v3/projects/1af43682b4f7494bbcff28102e0f93e2/users/26e573e68cae4f4aa52deb686c3ee239/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.838054 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3aee4c-0d06-4709-b8b8-dbb15ce6c503 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.840701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3aee4c-0d06-4709-b8b8-dbb15ce6c503 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.849696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1256/2532] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Aug 1 14:27:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:27:27.852001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af43682b4f7494bbcff28102e0f93e2, user_id=26e573e68cae4f4aa52deb686c3ee239, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:27.854658 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:27.868939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1277/2533] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:27 2022] PUT /identity/v3/projects/1af43682b4f7494bbcff28102e0f93e2/users/26e573e68cae4f4aa52deb686c3ee239/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:27.938957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:27.939243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:27.939551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:27.971174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] MFA Rules not processed for user `26e573e68cae4f4aa52deb686c3ee239`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:27.972416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:28.020261 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:28.045643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1257/2534] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:28.050145 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.116598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af226cfd-9eaa-4dc4-9202-a2ea5a9df949 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.121778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.122391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.122668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.122935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.123624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.125505 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.152497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1278/2535] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:28 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:28.156651 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.167736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cede3-a437-42bd-b9a2-47a9f453484d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.237349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.237981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.238268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.238549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.239225 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.241208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.263875 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1258/2536] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:28 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:28.267852 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.281497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34647b-fa08-4a01-85c3-3dfce582fec6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.286716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.287337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.287648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.287923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.288631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.290214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.301408 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1279/2537] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:28.305748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.316856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3954ab-3f11-4ee2-9ac1-9cb28646b5a1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.321732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.322420 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.322704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.322980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] PATH_INFO: `/v3/projects/96c59edcc1c342e0924048ddc0bb5393/users/44b0736cd3a64bea84718dc6c287739b/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.339140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c59edcc1c342e0924048ddc0bb5393, user_id=44b0736cd3a64bea84718dc6c287739b, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.341879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.355703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1259/2538] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:28 2022] PUT /identity/v3/projects/96c59edcc1c342e0924048ddc0bb5393/users/44b0736cd3a64bea84718dc6c287739b/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:28.359546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.369522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ce6b-1dc6-4d08-8302-d3e74520673f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.428382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554b602-53d1-4e8b-b558-04e0e478eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.429015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554b602-53d1-4e8b-b558-04e0e478eb13 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.429293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554b602-53d1-4e8b-b558-04e0e478eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.429568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554b602-53d1-4e8b-b558-04e0e478eb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.430270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554b602-53d1-4e8b-b558-04e0e478eb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.431904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554b602-53d1-4e8b-b558-04e0e478eb13 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.442489 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1280/2539] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:28.448535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.459236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b277209-b867-4b3e-ab4c-59915123cd9d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.463974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.464642 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.464959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.465248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] PATH_INFO: `/v3/projects/96c59edcc1c342e0924048ddc0bb5393/users/44b0736cd3a64bea84718dc6c287739b/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.467217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c59edcc1c342e0924048ddc0bb5393, user_id=44b0736cd3a64bea84718dc6c287739b, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.470310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.481893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1260/2540] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:28 2022] PUT /identity/v3/projects/96c59edcc1c342e0924048ddc0bb5393/users/44b0736cd3a64bea84718dc6c287739b/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:28.492168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.492480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.492753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.522198 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] MFA Rules not processed for user `44b0736cd3a64bea84718dc6c287739b`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:28.523683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:28.533949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:28.596623 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1281/2541] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:28.608157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:28.609895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:28.670540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180dae7-8726-4cdb-963c-6c582f6daf17 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:28.674139 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:28.674757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:28.675036 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:28.675308 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:28.724339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:28.736171 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:28.737632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:28.747684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1261/2542] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:30.172585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:30.174435 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:30.266665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc726a6-05b8-48b2-8300-fda9e0455ea2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:30.279810 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:30.281101 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:30.281573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:30.281989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:30.284364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:30.370217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:30.372620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:30.384556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1282/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:33.517610 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:33.519068 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:33.531821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06093e9-68cc-4793-a403-0940baa57c4d service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:33.535930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:33.536407 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:33.536722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:33.537341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:33.547504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:33.548979 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:33.560885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1262/2544] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:35.583920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:35.585432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:35.598605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9458a9-3b69-4a4e-9e9c-1c07216c02cf service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:35.603386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:35.604075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:35.604361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:35.604637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] PATH_INFO: `/v3/users/fbaa67d901e34419b1616a8457fbd683` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:35.605846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbaa67d901e34419b1616a8457fbd683)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:35.608046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:35.669541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbaa67d901e34419b1616a8457fbd683'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:35.670204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbaa67d901e34419b1616a8457fbd683'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:35.674791 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:35.685158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbaa67d901e34419b1616a8457fbd683'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:35.693135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbaa67d901e34419b1616a8457fbd683'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:35.701398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0db9241b-7f65-49b1-accd-c7cdf4c13006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbaa67d901e34419b1616a8457fbd683'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:35.707431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1283/2545] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:35 2022] DELETE /identity/v3/users/fbaa67d901e34419b1616a8457fbd683 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:35.897397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:35.912930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16739dd3-a129-4203-bc14-94cd7c6ce449 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:35.973696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:35.974230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:35.974509 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:35.974782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] PATH_INFO: `/v3/projects/2fb1590048254549bb480d677892031f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:35.975980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb1590048254549bb480d677892031f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:35.977914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.031910 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb1590048254549bb480d677892031f', 'request_id': 'req-8f969ec9-7c68-499f-a5c7-8547c18a422e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.038731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb1590048254549bb480d677892031f', 'request_id': 'req-8f969ec9-7c68-499f-a5c7-8547c18a422e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.049115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb1590048254549bb480d677892031f', 'request_id': 'req-8f969ec9-7c68-499f-a5c7-8547c18a422e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.063857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] The token cache is being invalidate because project 2fb1590048254549bb480d677892031f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:36.064168 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.064850 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f969ec9-7c68-499f-a5c7-8547c18a422e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.066499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1263/2546] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:35 2022] DELETE /identity/v3/projects/2fb1590048254549bb480d677892031f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:36.071602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.072032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.072358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.092877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:36.094347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:36.157377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1284/2547] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:36.165704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.165993 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.166259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.189399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:36.190898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:36.213107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1264/2548] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:36.218862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.230740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de805f5-af36-4050-8582-af0728c618d5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.239532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.240127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.240422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.240926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.241576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:36.242042 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.243778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.262224 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1285/2549] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:36.266072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.276309 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966d70-41b3-4beb-8841-7fec062da2e4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.285921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.287434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.287434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.287434 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.288333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.290388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.325118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1265/2550] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:36 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:36.331080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.346782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea90a56-c61e-4083-b102-797775e732e9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.426129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.427369 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.427885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.428516 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.429363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.431578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.458105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1286/2551] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:36 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:36.462470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.480861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ec716-af01-4cd1-b2bc-885c9966daf6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.492374 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.494193 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.494473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.494799 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.495549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.498686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.512499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1266/2552] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:36.516150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.528513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6ad63-7c45-423d-9ccf-396158bcad9f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.533981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136ee5e-1b47-4ef4-9bea-950b06cee058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.534746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136ee5e-1b47-4ef4-9bea-950b06cee058 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.535178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136ee5e-1b47-4ef4-9bea-950b06cee058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.535483 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136ee5e-1b47-4ef4-9bea-950b06cee058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.536491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136ee5e-1b47-4ef4-9bea-950b06cee058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.538097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136ee5e-1b47-4ef4-9bea-950b06cee058 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.560998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1287/2553] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:36.561527 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.573127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b66c9-a6ea-49ba-b55d-6ae5346db422 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.578731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.579435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.579749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.580028 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] PATH_INFO: `/v3/projects/99305bda3fe54f93a570e0786a0fcc03/users/96c44bd22d0145fd8836e4d2cd3f17a1/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.599012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99305bda3fe54f93a570e0786a0fcc03, user_id=96c44bd22d0145fd8836e4d2cd3f17a1, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.602029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.628613 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1267/2554] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:36 2022] PUT /identity/v3/projects/99305bda3fe54f93a570e0786a0fcc03/users/96c44bd22d0145fd8836e4d2cd3f17a1/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:36.638941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.639592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.639592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.678831 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] MFA Rules not processed for user `96c44bd22d0145fd8836e4d2cd3f17a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:36.680115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:36.707330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:36.737430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.740287 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:36.749897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:36.758865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1288/2555] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:36.771083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:36.772592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:36.815680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a33b0-6b9b-4f95-b1b0-75b89ecda93d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.820906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.821550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.821831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.822457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] PATH_INFO: `/v3/users/00393da6a11e4d77b5ad25b904f7187b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.823702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00393da6a11e4d77b5ad25b904f7187b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.825645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.850608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da4d1d-6940-437a-9e02-7185ae371f9b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:36.858821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:36.859551 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:36.859840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:36.860115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:36.869920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:36.872894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:36.881716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00393da6a11e4d77b5ad25b904f7187b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.882540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00393da6a11e4d77b5ad25b904f7187b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.883719 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1289/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:36.897752 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00393da6a11e4d77b5ad25b904f7187b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.907747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00393da6a11e4d77b5ad25b904f7187b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.908716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00393da6a11e4d77b5ad25b904f7187b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:36.939361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1268/2557] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:36 2022] DELETE /identity/v3/users/00393da6a11e4d77b5ad25b904f7187b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:37.168238 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:37.179684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91fd0a-d3cc-4435-99b2-fbd7f523dc48 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:37.210513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:37.212399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:37.248557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:37.249499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:37.250205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:37.250821 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] PATH_INFO: `/v3/users/96c44bd22d0145fd8836e4d2cd3f17a1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:37.252120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c44bd22d0145fd8836e4d2cd3f17a1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:37.254367 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:37.303822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866f727-77ae-4888-93fc-ac7197953451 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:37.308701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:37.309348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:37.309632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:37.309972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] PATH_INFO: `/v3/projects/83c251299d274471b80df4ff026b0102` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:37.310746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c44bd22d0145fd8836e4d2cd3f17a1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.311215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c251299d274471b80df4ff026b0102)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:37.311587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c44bd22d0145fd8836e4d2cd3f17a1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.313170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:37.316971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:37.327625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c44bd22d0145fd8836e4d2cd3f17a1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.336096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c44bd22d0145fd8836e4d2cd3f17a1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.345951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aa3b8132-e6ba-4850-855a-17d1d152f60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c44bd22d0145fd8836e4d2cd3f17a1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.349293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:37.352264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1290/2558] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:37 2022] DELETE /identity/v3/users/96c44bd22d0145fd8836e4d2cd3f17a1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:37.357526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:37.357789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:37.358030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:37.370653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c251299d274471b80df4ff026b0102', 'request_id': 'req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.376967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c251299d274471b80df4ff026b0102', 'request_id': 'req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.377115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:37.378105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:37.385552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c251299d274471b80df4ff026b0102', 'request_id': 'req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.397349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] The token cache is being invalidate because project 83c251299d274471b80df4ff026b0102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:37.397641 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.398289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.399842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1269/2559] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:37 2022] DELETE /identity/v3/projects/83c251299d274471b80df4ff026b0102 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:37.404384 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:37.405764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:37.444190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1291/2560] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:37.454909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:37.456394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:37.479803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397ca4b-217e-4fe0-8458-2a9a4f12edd6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:37.484506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:37.485269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:37.485530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:37.485727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] PATH_INFO: `/v3/users/55d498154b384496b6e0fe95511096f6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:37.487032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d498154b384496b6e0fe95511096f6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:37.489021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:37.523749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f5276-3e58-4840-9e79-51e7952591f6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:37.527999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:37.528631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:37.528924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:37.529241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:37.537820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d498154b384496b6e0fe95511096f6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.538651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:37.538929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d498154b384496b6e0fe95511096f6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.540191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:37.551948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d498154b384496b6e0fe95511096f6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.557877 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1292/2561] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:37.560185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d498154b384496b6e0fe95511096f6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.560952 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d498154b384496b6e0fe95511096f6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:37.577397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1270/2562] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:37 2022] DELETE /identity/v3/users/55d498154b384496b6e0fe95511096f6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:37.858680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:37.872293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59d203-fb5f-453f-802a-5bbd2795d031 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:37.922696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:37.924263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:37.946729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:37.947428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:37.947743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:37.948029 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] PATH_INFO: `/v3/projects/18402d0b12754d42a679dc9a70398e9b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:37.949285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18402d0b12754d42a679dc9a70398e9b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:37.951200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:37.968549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:37.971204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:38.001576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18402d0b12754d42a679dc9a70398e9b', 'request_id': 'req-3b1a05a6-681a-444b-b41b-e1507f5272b8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.006049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b7dd-877e-4300-86c7-75745a40adcd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.008495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18402d0b12754d42a679dc9a70398e9b', 'request_id': 'req-3b1a05a6-681a-444b-b41b-e1507f5272b8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.018885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18402d0b12754d42a679dc9a70398e9b', 'request_id': 'req-3b1a05a6-681a-444b-b41b-e1507f5272b8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.033056 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] The token cache is being invalidate because project 18402d0b12754d42a679dc9a70398e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:38.033270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.033915 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b1a05a6-681a-444b-b41b-e1507f5272b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.035652 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1293/2563] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:37 2022] DELETE /identity/v3/projects/18402d0b12754d42a679dc9a70398e9b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:38.040587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535018e9-f464-4a8f-9928-31ddc7e8b8ab None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.040872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535018e9-f464-4a8f-9928-31ddc7e8b8ab None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.041149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535018e9-f464-4a8f-9928-31ddc7e8b8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.061053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-535018e9-f464-4a8f-9928-31ddc7e8b8ab None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:38.062281 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-535018e9-f464-4a8f-9928-31ddc7e8b8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.068521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.069285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.069700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.070559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] PATH_INFO: `/v3/projects/99305bda3fe54f93a570e0786a0fcc03` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.071713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99305bda3fe54f93a570e0786a0fcc03)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.074020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.124745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1294/2564] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:38.132493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.132789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.133072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.133820 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99305bda3fe54f93a570e0786a0fcc03', 'request_id': 'req-47eae01f-e850-4b04-a352-251199d8316b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.139549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99305bda3fe54f93a570e0786a0fcc03', 'request_id': 'req-47eae01f-e850-4b04-a352-251199d8316b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.147209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99305bda3fe54f93a570e0786a0fcc03', 'request_id': 'req-47eae01f-e850-4b04-a352-251199d8316b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.155279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:38.156555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.159177 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] The token cache is being invalidate because project 99305bda3fe54f93a570e0786a0fcc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:38.159516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.160209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-47eae01f-e850-4b04-a352-251199d8316b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:38.162461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1271/2565] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:37 2022] DELETE /identity/v3/projects/99305bda3fe54f93a570e0786a0fcc03 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:38.167283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419ddb7-657c-42c2-90fd-f825e2220c3b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.168186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419ddb7-657c-42c2-90fd-f825e2220c3b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.168476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419ddb7-657c-42c2-90fd-f825e2220c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.189332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1419ddb7-657c-42c2-90fd-f825e2220c3b None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:38.190643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1419ddb7-657c-42c2-90fd-f825e2220c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.219121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1295/2566] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:38.223929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.232777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb19f2-d2e2-479f-8e66-756733004478 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.237607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.238229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.238514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.238789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.239515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:38.239959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.241580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.253613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1296/2567] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:27:38.254724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1272/2568] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:38.258304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.264159 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.264477 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.264757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.268888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c05b4-64be-4cd2-a2fe-488b18198051 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.273563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.274172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.274455 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.274744 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.275386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.277882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.287281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:38.288543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.296804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1273/2569] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:38.301058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.306841 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:38.310773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1297/2570] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:38.311094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f9ab2-24df-49bf-8d12-333435b2b4f2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.315176 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.330199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:38.384355 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.385145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.385427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.385701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.387540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:38.387862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.389498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.394600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a638-5722-4056-8dd7-64c7b74abd04 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.399085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.399991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.400466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.400936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.401711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.404363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.407430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1274/2571] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:27:38.412103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.425137 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1057f-3bbe-4bc0-ba15-6c89906d90b1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.432521 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.433186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.433478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.433761 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.434430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.439796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.440549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1298/2572] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:38.441790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.453031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6762c3a-d126-41c5-ab07-36aea65021bb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.458839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.459610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.459955 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.460307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.461090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.462684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.474770 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1299/2573] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:38.478843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1275/2574] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:38.478983 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.482523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.490524 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421fd63-e948-4082-b280-7e89b62573a8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.561229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.561600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7199a-f409-4d7e-8e1d-1465bbf73dda admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.561994 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.562122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.562299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.563008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.564637 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.567803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.568460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.568745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.569024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.569740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.571757 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.578366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1300/2575] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:38.589156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.599186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1276/2576] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:38.603323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.606128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a054-95e2-4757-be42-944c755ef139 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.612481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2e635-bac3-489b-ab1f-14f97d150e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.613433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e635-bac3-489b-ab1f-14f97d150e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.613722 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e635-bac3-489b-ab1f-14f97d150e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.614001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e635-bac3-489b-ab1f-14f97d150e40 admin admin] PATH_INFO: `/v3/projects/47e500c5985743a2a5abfb99c0f42ab7/users/540bec8413b144889a07e9d9458b298e/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.618285 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6493b-af06-4dc7-8eec-9896bedcb4c1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.623891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.624496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.624913 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.625202 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.625964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.627678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.633919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2e635-bac3-489b-ab1f-14f97d150e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e500c5985743a2a5abfb99c0f42ab7, user_id=540bec8413b144889a07e9d9458b298e, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.637444 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2e635-bac3-489b-ab1f-14f97d150e40 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.639487 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1277/2577] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:38.643873 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.655209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192b85-3bb4-4063-b6b8-8230bbc6067c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.658338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1301/2578] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:38 2022] PUT /identity/v3/projects/47e500c5985743a2a5abfb99c0f42ab7/users/540bec8413b144889a07e9d9458b298e/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:38.659916 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.660227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.660493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.679967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:38.681321 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.710678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:38.720600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.721218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.721497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.721767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.722462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.724120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.734458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:38.735627 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1278/2579] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:38.739430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.745122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1302/2580] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:38.749864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7003-759e-4040-b607-9931a0aafc80 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.754673 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91ab0c-e7b0-4cf3-8aee-6cdd5a816d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.755331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91ab0c-e7b0-4cf3-8aee-6cdd5a816d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.755451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.755643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91ab0c-e7b0-4cf3-8aee-6cdd5a816d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.755920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91ab0c-e7b0-4cf3-8aee-6cdd5a816d8e admin admin] PATH_INFO: `/v3/projects/74babf795eb546dda7390d6443b1f95f/users/c0cd785561f748c69718b4560ca8cd54/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.756889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.771557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91ab0c-e7b0-4cf3-8aee-6cdd5a816d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74babf795eb546dda7390d6443b1f95f, user_id=c0cd785561f748c69718b4560ca8cd54, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.774216 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91ab0c-e7b0-4cf3-8aee-6cdd5a816d8e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.788797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1279/2581] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:38 2022] PUT /identity/v3/projects/74babf795eb546dda7390d6443b1f95f/users/c0cd785561f748c69718b4560ca8cd54/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:38.800416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.800823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.801107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.819380 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5413-c51b-4883-b232-9e4958ae1620 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.829606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] MFA Rules not processed for user `c0cd785561f748c69718b4560ca8cd54`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:38.830822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:38.872231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.872710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.872992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.873265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.897468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1280/2582] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:38.902885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:38.938634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.940051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:38.950066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1303/2583] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:38.971142 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473435b-e21b-45f0-a182-69d10f596d90 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:38.976428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:38.977166 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:38.977456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:38.977763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:38.978445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:38.980932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.009240 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1281/2584] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:38 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:39.013610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.023703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e152d-2789-4af3-9dec-2f97782cc0dd service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.088334 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.088952 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.089236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.089528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.090231 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.092184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.103803 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:39.117213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1304/2585] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:39 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:39.122528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.133858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ac5ad-ac47-4915-95a1-366f724c58d5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.140638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.141391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.141549 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.141838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.142462 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.144169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.155313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1282/2586] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:39.159963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.171199 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add60c4-93b2-47ba-8527-42b1c737d2ae admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.175999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.176680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.176967 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.177247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] PATH_INFO: `/v3/projects/baa9a443a8454b548aaad15a2431549c/users/015059528a6349488ab14e44e2f7ab3b/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.195160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa9a443a8454b548aaad15a2431549c, user_id=015059528a6349488ab14e44e2f7ab3b, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.197863 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.211815 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1305/2587] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:39 2022] PUT /identity/v3/projects/baa9a443a8454b548aaad15a2431549c/users/015059528a6349488ab14e44e2f7ab3b/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:39.215965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.226890 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bd26f-828b-4d9f-8558-d335ec22603d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.260661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:39.296958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845bfc-717c-42df-b409-191e3dd22166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.297986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845bfc-717c-42df-b409-191e3dd22166 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.298385 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845bfc-717c-42df-b409-191e3dd22166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.298787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845bfc-717c-42df-b409-191e3dd22166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.299749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2845bfc-717c-42df-b409-191e3dd22166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.301843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2845bfc-717c-42df-b409-191e3dd22166 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.315754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1283/2588] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:39.319170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.331588 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ce222-e04f-464c-aa81-c312e1b6fc71 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.336946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.337648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.337929 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.338245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] PATH_INFO: `/v3/projects/baa9a443a8454b548aaad15a2431549c/users/015059528a6349488ab14e44e2f7ab3b/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.340871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa9a443a8454b548aaad15a2431549c, user_id=015059528a6349488ab14e44e2f7ab3b, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.343951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.356485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1306/2589] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:39 2022] PUT /identity/v3/projects/baa9a443a8454b548aaad15a2431549c/users/015059528a6349488ab14e44e2f7ab3b/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:39.368117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.368455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.368681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.402831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] MFA Rules not processed for user `015059528a6349488ab14e44e2f7ab3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:39.404254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:39.496144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1284/2590] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:39.507569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.510509 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:39.583091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cb746-3f5b-4200-9412-6fd80c52cd6f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.587172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2fc7b-8661-4c0f-9235-00b2feff9848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.587945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2fc7b-8661-4c0f-9235-00b2feff9848 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.588123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2fc7b-8661-4c0f-9235-00b2feff9848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.588423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2fc7b-8661-4c0f-9235-00b2feff9848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.599577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2fc7b-8661-4c0f-9235-00b2feff9848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.601144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2fc7b-8661-4c0f-9235-00b2feff9848 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.613141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1307/2591] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:39.719863 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:39.731429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed749167-07cf-4339-9bab-63ca8b47bc1f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:39.736425 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:39.737211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:39.737733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:39.737867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:39.816191 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:39.817703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:39.827963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1285/2592] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:43.575274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:43.575996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:43.575996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:43.610348 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] MFA Rules not processed for user `540bec8413b144889a07e9d9458b298e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:43.612058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:43.712195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1308/2593] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:43.717107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:43.781241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:43.800143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ff5a5-93c5-4a2b-b7e8-d6e3f90fecdb service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:43.805387 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:43.806380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:43.806866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:43.807141 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:43.808508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:43.810467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:43.850567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1286/2594] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:43 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:43.852228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:43.866424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d62b5-70be-4176-9eb8-dcb09edadd89 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:43.919069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:43.931106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:43.931773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:43.932055 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:43.932329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:43.933537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:43.935496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:43.958765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1309/2595] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:43 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:43.963550 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:43.977131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5227b-feda-40b5-8718-fa6cb6d35e6c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:43.982277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:43.982936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:43.983221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:43.983552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:43.984260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:43.985902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:43.997886 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1287/2596] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:44.002084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:44.013582 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67977e-a6ec-4f4e-a4a5-826a2913a501 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:44.019622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:44.020372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:44.020606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:44.020886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] PATH_INFO: `/v3/projects/b586502dc95e4aa4af33b77d3466cb92/users/c0d20c41cd784174b7890a788a094f37/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:44.041886 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b586502dc95e4aa4af33b77d3466cb92, user_id=c0d20c41cd784174b7890a788a094f37, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:44.044696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:44.064039 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1310/2597] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:44 2022] PUT /identity/v3/projects/b586502dc95e4aa4af33b77d3466cb92/users/c0d20c41cd784174b7890a788a094f37/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:44.065268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:44.077325 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d78df-4385-44c9-bdf3-ab7bed133e70 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:44.141160 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f6971-4d8e-4f3e-b82f-1b04aa35d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:44.141775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f6971-4d8e-4f3e-b82f-1b04aa35d72a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:44.142057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f6971-4d8e-4f3e-b82f-1b04aa35d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:44.142328 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f6971-4d8e-4f3e-b82f-1b04aa35d72a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:44.143023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f6971-4d8e-4f3e-b82f-1b04aa35d72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:44.144936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f6971-4d8e-4f3e-b82f-1b04aa35d72a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:44.155517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1288/2598] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:44.160926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:44.173474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578703-ec2c-4e78-ad7e-2a4241a8a46e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:44.178341 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:44.179010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:44.179305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:44.179634 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] PATH_INFO: `/v3/projects/b586502dc95e4aa4af33b77d3466cb92/users/c0d20c41cd784174b7890a788a094f37/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:44.182034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b586502dc95e4aa4af33b77d3466cb92, user_id=c0d20c41cd784174b7890a788a094f37, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:44.185004 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:44.198311 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1311/2599] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:44 2022] PUT /identity/v3/projects/b586502dc95e4aa4af33b77d3466cb92/users/c0d20c41cd784174b7890a788a094f37/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:48.614211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:48.615041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:48.615041 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:48.648571 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] MFA Rules not processed for user `c0d20c41cd784174b7890a788a094f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:48.649828 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:48.740981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1289/2600] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:48.757971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:48.759935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:48.844009 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad289118-7c13-4c9f-ac73-572b183e7437 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:48.850065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f3514-b367-410b-9de2-b361f0f9a339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:48.850785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f3514-b367-410b-9de2-b361f0f9a339 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:48.851092 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f3514-b367-410b-9de2-b361f0f9a339 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:48.851390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f3514-b367-410b-9de2-b361f0f9a339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:48.920280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f3514-b367-410b-9de2-b361f0f9a339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:48.921971 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f3514-b367-410b-9de2-b361f0f9a339 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:48.937047 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1312/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:52.305219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:52.368379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:52.386117 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a83c1f-8b3f-49b7-805e-43771b95632e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:52.390970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:52.391667 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:52.391956 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:52.392235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] PATH_INFO: `/v3/users/c0cd785561f748c69718b4560ca8cd54` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:52.393448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0cd785561f748c69718b4560ca8cd54)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:52.395448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:52.459864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0cd785561f748c69718b4560ca8cd54'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:52.460605 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0cd785561f748c69718b4560ca8cd54'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:52.475034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0cd785561f748c69718b4560ca8cd54'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:52.486284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0cd785561f748c69718b4560ca8cd54'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:52.486284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0cd785561f748c69718b4560ca8cd54'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:52.501110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1290/2602] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:52 2022] DELETE /identity/v3/users/c0cd785561f748c69718b4560ca8cd54 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:52.506580 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:52.507098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:52.507270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:52.529160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:52.530702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:52.570302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:52.605033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1313/2603] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:52.616208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:52.618107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:52.694162 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe209-e37b-481b-bd49-dc3515fb5779 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:52.697926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:52.699092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:52.699375 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:52.699789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:52.709789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:52.711248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:52.722957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1291/2604] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:27:53.029534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:53.032164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:53.047865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b35f-8f19-4114-8185-572cc2adbe02 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:53.053146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:53.053881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:53.054248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:53.054447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] PATH_INFO: `/v3/projects/74babf795eb546dda7390d6443b1f95f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:53.055732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] RBAC: Authorizing `identity:delete_project(project_id=74babf795eb546dda7390d6443b1f95f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:53.058257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:53.126888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74babf795eb546dda7390d6443b1f95f', 'request_id': 'req-f82d7e1d-f40f-4259-9a32-4e62adfb539c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.132641 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74babf795eb546dda7390d6443b1f95f', 'request_id': 'req-f82d7e1d-f40f-4259-9a32-4e62adfb539c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.145539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74babf795eb546dda7390d6443b1f95f', 'request_id': 'req-f82d7e1d-f40f-4259-9a32-4e62adfb539c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.157478 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] The token cache is being invalidate because project 74babf795eb546dda7390d6443b1f95f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:53.157766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.158522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.160020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1314/2605] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:53 2022] DELETE /identity/v3/projects/74babf795eb546dda7390d6443b1f95f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:53.164324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:53.176348 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcef83d-1dce-4989-bf45-0d9acc42db8a service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:53.239312 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:53.240017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:53.240321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:53.240587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] PATH_INFO: `/v3/users/015059528a6349488ab14e44e2f7ab3b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:53.241996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=015059528a6349488ab14e44e2f7ab3b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:53.244388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:53.257087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:53.428169 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015059528a6349488ab14e44e2f7ab3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.428852 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015059528a6349488ab14e44e2f7ab3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.441688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015059528a6349488ab14e44e2f7ab3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.448543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015059528a6349488ab14e44e2f7ab3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.449218 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93055395-7ce8-457a-b568-23b3c2f0b77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015059528a6349488ab14e44e2f7ab3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.462671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1292/2606] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:53 2022] DELETE /identity/v3/users/015059528a6349488ab14e44e2f7ab3b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:53.785783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:53.785783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:53.799275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7e1d-f40f-4259-9a32-4e62adfb539c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:53.862179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:53.862747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:53.863030 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:53.863307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] PATH_INFO: `/v3/projects/baa9a443a8454b548aaad15a2431549c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:53.864590 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa9a443a8454b548aaad15a2431549c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:53.866510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:53.914793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa9a443a8454b548aaad15a2431549c', 'request_id': 'req-78bb96db-f497-4576-85b1-3ea546ad6881'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.921128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa9a443a8454b548aaad15a2431549c', 'request_id': 'req-78bb96db-f497-4576-85b1-3ea546ad6881'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.928742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa9a443a8454b548aaad15a2431549c', 'request_id': 'req-78bb96db-f497-4576-85b1-3ea546ad6881'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.930078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:53.943497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] The token cache is being invalidate because project baa9a443a8454b548aaad15a2431549c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:53.943788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.944448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-78bb96db-f497-4576-85b1-3ea546ad6881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:53.946024 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1315/2607] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:53 2022] DELETE /identity/v3/projects/baa9a443a8454b548aaad15a2431549c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:53.951596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:53.951892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:53.952174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:53.972483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:53.973703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:54.035219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1293/2608] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:54.042995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.043496 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.043784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.063771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:54.065020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:54.073685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1316/2609] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:54.078379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.088415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d41cb-f4e2-4ab2-b220-d8eee303aa35 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.093957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.094570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.094902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.095174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.095912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:54.096220 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.097814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.113399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1294/2610] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:54.114541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.123787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb483dc8-70ec-4099-bcdf-9d8fe6e39000 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.128471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.129175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.129514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.129843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.130568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.132576 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.161727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1317/2611] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:54.166450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.177857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f0e31-6c8a-43ec-bcd0-342ffe51873e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.237200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.237825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.238113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.238403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.239067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.241016 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.262454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1295/2612] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:54.266626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.280502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63cc17-9aba-4b2e-8bd7-5ca93d98a416 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.285703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.286331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.286611 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.286887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.287626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.289312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.300267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1318/2613] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:54.304363 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.311279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:54.315380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510eb9d-0022-434a-a002-af472698f017 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.321581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d83dd-b809-4144-a000-c6b0604c78fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.322296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d83dd-b809-4144-a000-c6b0604c78fd admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.322764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d83dd-b809-4144-a000-c6b0604c78fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.323835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d83dd-b809-4144-a000-c6b0604c78fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.324603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d83dd-b809-4144-a000-c6b0604c78fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.326381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d83dd-b809-4144-a000-c6b0604c78fd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.336963 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1296/2614] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:54.341326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.351950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1c737-0f91-46bc-bf80-38b411e9c9ad admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.356753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.357451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.357737 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.358015 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] PATH_INFO: `/v3/projects/b73db8af23da43a7bbb7653587a597ea/users/cc9a5809e8dc473087b416394e5659c7/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.374953 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73db8af23da43a7bbb7653587a597ea, user_id=cc9a5809e8dc473087b416394e5659c7, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.377583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.391846 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1319/2615] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:54 2022] PUT /identity/v3/projects/b73db8af23da43a7bbb7653587a597ea/users/cc9a5809e8dc473087b416394e5659c7/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:54.403002 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.403270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.403375 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.433871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] MFA Rules not processed for user `cc9a5809e8dc473087b416394e5659c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:54.435082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:54.502272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1297/2616] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:27:54.507034 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.572665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016e141-ff64-4ce9-9f3c-4a7223899b3b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.578063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.578678 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.578963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.579241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.579939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.581833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.611494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1320/2617] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:54.614242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.625833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea249368-2ea5-4e3f-a77e-59070267b62e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.687743 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.688369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.688655 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.688932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.689614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.691598 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.718533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1298/2618] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:54.719521 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.730499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec7357-c4a4-4c4f-810b-9d010688a9f7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.735490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.736166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.736457 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.736730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.737451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.739067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.750986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1321/2619] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:27:54.755552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.766341 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbaa76-eaa1-49f1-91b7-431719be368f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.771809 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.772472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.772749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.773351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] PATH_INFO: `/v3/projects/f50bc44aba1d4958bb2404efa79b2750/users/88679ad59d824155bd3c5ae73505422b/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.788864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50bc44aba1d4958bb2404efa79b2750, user_id=88679ad59d824155bd3c5ae73505422b, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.791634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.806143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1299/2620] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:54 2022] PUT /identity/v3/projects/f50bc44aba1d4958bb2404efa79b2750/users/88679ad59d824155bd3c5ae73505422b/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:27:54.810360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.820665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b50ddd-eb15-4cae-81c6-d38432ed4443 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.878459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53734940-d890-484e-86b9-304c53bde5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.879115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53734940-d890-484e-86b9-304c53bde5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.879449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53734940-d890-484e-86b9-304c53bde5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.879816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53734940-d890-484e-86b9-304c53bde5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.880618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53734940-d890-484e-86b9-304c53bde5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.882276 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53734940-d890-484e-86b9-304c53bde5b1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.887989 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:54.893545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1322/2621] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:54.897401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:54.914511 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22926c14-04e1-4002-8010-cd7ed4fe0b1f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:54.919563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:54.920247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.920529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.920806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] PATH_INFO: `/v3/projects/f50bc44aba1d4958bb2404efa79b2750/users/88679ad59d824155bd3c5ae73505422b/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.923090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50bc44aba1d4958bb2404efa79b2750, user_id=88679ad59d824155bd3c5ae73505422b, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:54.925787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:54.937077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1300/2622] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:54 2022] PUT /identity/v3/projects/f50bc44aba1d4958bb2404efa79b2750/users/88679ad59d824155bd3c5ae73505422b/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:54.950403 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:54.950711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:54.950984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:54.986889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] MFA Rules not processed for user `88679ad59d824155bd3c5ae73505422b`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:54.988178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:55.060827 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1323/2623] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:55.073825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:55.075661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:55.118053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:55.138278 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcc03d-4d5c-4020-b585-1b9005239bea admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:55.142103 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:55.142734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:55.143020 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:55.143300 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:55.153072 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:55.154442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:55.164552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1301/2624] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:55.490243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:55.500721 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194d77-0943-4502-855f-7b938d36d4df None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:55.506755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88679ad59d824155bd3c5ae73505422b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f50bc44aba1d4958bb2404efa79b2750', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:55.509390 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:55.509686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:55.509991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:55.510777 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:55.511058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:55.512731 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:55.528972 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1324/2625] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:55.530083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:55.539717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13932ec5-d0a8-4800-a578-6803771b45f9 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:55.545147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88679ad59d824155bd3c5ae73505422b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f50bc44aba1d4958bb2404efa79b2750', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:55.545810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:55.546150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:55.546372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:55.547494 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:55.549495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:55.583788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1302/2626] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:55 2022] POST /identity/v3/projects => generated 354 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:55.587350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:55.597774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836576-e819-4432-b111-0eb511d5dda0 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:55.662761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88679ad59d824155bd3c5ae73505422b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f50bc44aba1d4958bb2404efa79b2750', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:55.663397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:55.663712 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:55.663993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:55.664693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:55.665005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:55.666662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:55.680707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1325/2627] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:55.718773 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:55.729102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477468a8-f3af-45fe-85a9-c72712a61a6e tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:55.734476 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88679ad59d824155bd3c5ae73505422b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f50bc44aba1d4958bb2404efa79b2750', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:55.735227 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:55.735614 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:55.735964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] PATH_INFO: `/v3/projects/f34b258b8fdd4a6baaf671c228988b79` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:55.737161 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f34b258b8fdd4a6baaf671c228988b79)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:55.739082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:55.799597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34b258b8fdd4a6baaf671c228988b79', 'request_id': 'req-03643f0f-5474-4f09-a2b9-a1b668f27164'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:55.806846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34b258b8fdd4a6baaf671c228988b79', 'request_id': 'req-03643f0f-5474-4f09-a2b9-a1b668f27164'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:55.821094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34b258b8fdd4a6baaf671c228988b79', 'request_id': 'req-03643f0f-5474-4f09-a2b9-a1b668f27164'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:55.846442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] The token cache is being invalidate because project f34b258b8fdd4a6baaf671c228988b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:55.847004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:55.847994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-03643f0f-5474-4f09-a2b9-a1b668f27164 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:55.850565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1303/2628] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:55 2022] DELETE /identity/v3/projects/f34b258b8fdd4a6baaf671c228988b79 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:56.047236 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:56.049201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:56.116128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c475e-fa94-49a2-93b5-d0a7a5428a90 tempest-VolumeQuotaClassesTest-1454659372 tempest-VolumeQuotaClassesTest-1454659372-project-admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:56.124813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:56.177700 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:56.178284 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:56.178567 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:56.178844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] PATH_INFO: `/v3/users/cc9a5809e8dc473087b416394e5659c7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:56.180445 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc9a5809e8dc473087b416394e5659c7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:56.183280 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:56.235695 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc9a5809e8dc473087b416394e5659c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.237142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc9a5809e8dc473087b416394e5659c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.257635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc9a5809e8dc473087b416394e5659c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.266377 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc9a5809e8dc473087b416394e5659c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.273839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc9a5809e8dc473087b416394e5659c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.285434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1326/2629] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:56 2022] DELETE /identity/v3/users/cc9a5809e8dc473087b416394e5659c7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:56.289936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:56.290233 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:56.290497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:56.309394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:56.310635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:56.382765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1304/2630] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:56.393100 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:56.394536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:56.454213 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9dbb0-a7a5-4c22-b799-491855a3a366 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:56.457794 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:56.458415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:56.458702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:56.458978 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:56.469166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:56.470625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:56.480495 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1327/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:56.767732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:56.769188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:56.781432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2e0a5-ee1d-46b9-bd92-d3ebb0e52ac7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:56.787499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:56.787499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:56.787499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:56.787499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] PATH_INFO: `/v3/projects/b73db8af23da43a7bbb7653587a597ea` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:56.788866 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73db8af23da43a7bbb7653587a597ea)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:56.797219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:56.830805 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:56.845215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73db8af23da43a7bbb7653587a597ea', 'request_id': 'req-629f3bea-4099-48ae-929c-16ec90b57084'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.851923 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73db8af23da43a7bbb7653587a597ea', 'request_id': 'req-629f3bea-4099-48ae-929c-16ec90b57084'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.859915 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73db8af23da43a7bbb7653587a597ea', 'request_id': 'req-629f3bea-4099-48ae-929c-16ec90b57084'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.874518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] The token cache is being invalidate because project b73db8af23da43a7bbb7653587a597ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:56.874810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.875502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:56.877004 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1305/2632] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:56 2022] DELETE /identity/v3/projects/b73db8af23da43a7bbb7653587a597ea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:56.885254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:56.894845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c45ff-d2ab-4dca-94c5-ea346ce9a430 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:56.931112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:56.959405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:56.960073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:56.960351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:56.960673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] PATH_INFO: `/v3/users/88679ad59d824155bd3c5ae73505422b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:56.961833 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88679ad59d824155bd3c5ae73505422b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:56.964420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.012494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88679ad59d824155bd3c5ae73505422b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.013177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88679ad59d824155bd3c5ae73505422b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.025008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88679ad59d824155bd3c5ae73505422b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.033419 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88679ad59d824155bd3c5ae73505422b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.040485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1f661c1f-b885-4511-b1aa-a950af25c1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88679ad59d824155bd3c5ae73505422b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.046456 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1328/2633] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:27:56 2022] DELETE /identity/v3/users/88679ad59d824155bd3c5ae73505422b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:57.324100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.325556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:57.339857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f3bea-4099-48ae-929c-16ec90b57084 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.404097 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.404762 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.405048 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.405323 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] PATH_INFO: `/v3/projects/f50bc44aba1d4958bb2404efa79b2750` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.406512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50bc44aba1d4958bb2404efa79b2750)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.408466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.460840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50bc44aba1d4958bb2404efa79b2750', 'request_id': 'req-b0ceb72b-a3cf-4f4c-b346-121536455d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.470876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50bc44aba1d4958bb2404efa79b2750', 'request_id': 'req-b0ceb72b-a3cf-4f4c-b346-121536455d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.481730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50bc44aba1d4958bb2404efa79b2750', 'request_id': 'req-b0ceb72b-a3cf-4f4c-b346-121536455d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.494594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] The token cache is being invalidate because project f50bc44aba1d4958bb2404efa79b2750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:27:57.494594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.496080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b0ceb72b-a3cf-4f4c-b346-121536455d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:27:57.497801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1306/2634] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:27:57 2022] DELETE /identity/v3/projects/f50bc44aba1d4958bb2404efa79b2750 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:57.503662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.504173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.504591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.526800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:57.528486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:57.600928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1329/2635] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:57.608897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.609184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.609460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.630690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:57.632600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:57.644543 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1307/2636] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:57.646944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.660881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a5cb7-ee87-405b-b486-fe606a4c7108 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.666288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.667589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.667871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.668163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.669104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:27:57.669405 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.671070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.692450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1330/2637] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:27:57.696964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.707399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ecb4e-4f40-4880-b416-175da7a96a17 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.712172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.712794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.713074 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.713349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.714609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.716643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.747636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1308/2638] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:57 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:57.752173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.763363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06a151-5a32-4355-b85c-434ac4bd9c26 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.808959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:57.835119 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.835839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.836122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.836416 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.837084 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.839693 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.867132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1331/2639] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:57 2022] POST /identity/v3/users => generated 301 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:57.868533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.880650 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1167c1e-9065-4fd9-911d-567a60607ac7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.886107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.886709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.887075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.887249 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.887989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.889689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.900428 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1309/2640] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:57.905233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.915555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e4b53-b65b-4a37-9313-8de5559b740f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.921060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7635623-3191-46af-ae7b-636f03b23664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.921861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7635623-3191-46af-ae7b-636f03b23664 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.922366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7635623-3191-46af-ae7b-636f03b23664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.922946 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7635623-3191-46af-ae7b-636f03b23664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.923932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7635623-3191-46af-ae7b-636f03b23664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.925542 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7635623-3191-46af-ae7b-636f03b23664 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.936581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1332/2641] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:57.943700 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:57.954253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8924e67-302a-49fc-a2a5-97b7eb7d07ec admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:57.958858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:57.959542 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:57.959821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:57.960089 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] PATH_INFO: `/v3/projects/ca5366bbbf864229979f8a9f936a1897/users/306f2dcf3ec0498995b044113edbcb0c/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:57.973154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:57.977865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5366bbbf864229979f8a9f936a1897, user_id=306f2dcf3ec0498995b044113edbcb0c, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:57.982199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:57.997555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1310/2642] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:57 2022] PUT /identity/v3/projects/ca5366bbbf864229979f8a9f936a1897/users/306f2dcf3ec0498995b044113edbcb0c/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:58.008085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.008375 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.008651 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.037506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] MFA Rules not processed for user `306f2dcf3ec0498995b044113edbcb0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:58.038709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:58.106688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1333/2643] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:58.111580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.176211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758cf71-f13e-435f-8bc9-9fd7483a6c1b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.180929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.181574 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.181857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.182138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.183339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.185277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.213935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1311/2644] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:27:58 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:27:58.218368 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.229698 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9eced-49f2-4bd4-a254-47a168824b19 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.444026 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.444608 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.444914 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.445400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.446439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.452073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.474975 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1334/2645] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:27:58 2022] POST /identity/v3/users => generated 307 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:27:58.484526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.495865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc98ad-1092-4d6f-a323-25abadc38942 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.500565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.501184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.501470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.501742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.502439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.504102 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.515311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1312/2646] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:58.519992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.530261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a5855-db32-4b90-98b3-812bdd304d3f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.534842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.535545 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.535826 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.536098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] PATH_INFO: `/v3/projects/cf15717a282045b2829b8de0d8fe4332/users/9403ea7325d2406985fe344858baa7e0/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.550980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf15717a282045b2829b8de0d8fe4332, user_id=9403ea7325d2406985fe344858baa7e0, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.553840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.568892 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1335/2647] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:58 2022] PUT /identity/v3/projects/cf15717a282045b2829b8de0d8fe4332/users/9403ea7325d2406985fe344858baa7e0/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:58.573091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.583144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c330eb1-2794-4075-b9a2-9c1a6b7dacf1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.642248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e57d-8253-4962-9baa-bd03c9d901db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.642870 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e57d-8253-4962-9baa-bd03c9d901db admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.643151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e57d-8253-4962-9baa-bd03c9d901db admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.643517 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e57d-8253-4962-9baa-bd03c9d901db admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.644253 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e57d-8253-4962-9baa-bd03c9d901db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.645848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e57d-8253-4962-9baa-bd03c9d901db admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.659959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1313/2648] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:27:58.660933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.672934 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f375d1-21ce-41de-ad7e-57ea8f63f748 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.677948 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.678526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.678805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.679079 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] PATH_INFO: `/v3/projects/cf15717a282045b2829b8de0d8fe4332/users/9403ea7325d2406985fe344858baa7e0/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.681208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf15717a282045b2829b8de0d8fe4332, user_id=9403ea7325d2406985fe344858baa7e0, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.684424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.695094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1336/2649] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:27:58 2022] PUT /identity/v3/projects/cf15717a282045b2829b8de0d8fe4332/users/9403ea7325d2406985fe344858baa7e0/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:27:58.706570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.706862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.707153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.739423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] MFA Rules not processed for user `9403ea7325d2406985fe344858baa7e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:27:58.740764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:58.794413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:58.816824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1314/2650] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:27:58.827874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:58.829583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:58.834960 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:27:58.897968 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e22fe6-59ca-43a1-9f18-fcfb954de93f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:58.902581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:58.903197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:58.903504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:58.903783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:58.913103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:58.914531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:58.925997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1337/2651] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:58 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:27:59.031073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:27:59.032586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:27:59.045631 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe28084-a329-4a03-9bd0-de44c8c37b4a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:27:59.049887 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce2f85-4b51-4370-ac01-b437091a055b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:27:59.050788 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce2f85-4b51-4370-ac01-b437091a055b service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:27:59.051079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce2f85-4b51-4370-ac01-b437091a055b service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:27:59.051353 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce2f85-4b51-4370-ac01-b437091a055b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:27:59.110843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce2f85-4b51-4370-ac01-b437091a055b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:27:59.112314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce2f85-4b51-4370-ac01-b437091a055b service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:27:59.121720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1315/2652] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:00.092434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:00.094241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:00.159356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e54f6d-4a74-4370-a277-19fc4b070203 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:00.164135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:00.164796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:00.165082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:00.165363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] PATH_INFO: `/v3/users/26e573e68cae4f4aa52deb686c3ee239` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:00.166785 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e573e68cae4f4aa52deb686c3ee239)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:00.168761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:00.220705 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e573e68cae4f4aa52deb686c3ee239'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.221412 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e573e68cae4f4aa52deb686c3ee239'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.233491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e573e68cae4f4aa52deb686c3ee239'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.241021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e573e68cae4f4aa52deb686c3ee239'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.248048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e573e68cae4f4aa52deb686c3ee239'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.253940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1338/2653] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:00 2022] DELETE /identity/v3/users/26e573e68cae4f4aa52deb686c3ee239 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:00.259056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:00.259621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:00.259731 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:00.280523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:00.281864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:00.347858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1316/2654] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:00.357689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:00.359142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:00.389597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:00.423073 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868304fd-ea93-46f9-a692-00254e316b0f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:00.426690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:00.427312 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:00.427645 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:00.427926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:00.437819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:00.439240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:00.450241 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1339/2655] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:00.767123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:00.769533 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:00.785625 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3fa6e-52dc-47cd-9bf7-2d099f1fa9dd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:00.791666 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:00.793349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:00.793734 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:00.794286 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] PATH_INFO: `/v3/projects/1af43682b4f7494bbcff28102e0f93e2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:00.796349 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af43682b4f7494bbcff28102e0f93e2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:00.802847 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:00.851380 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af43682b4f7494bbcff28102e0f93e2', 'request_id': 'req-10d6ef1c-b777-4393-8e76-75eec29692bf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.858469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af43682b4f7494bbcff28102e0f93e2', 'request_id': 'req-10d6ef1c-b777-4393-8e76-75eec29692bf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.866861 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af43682b4f7494bbcff28102e0f93e2', 'request_id': 'req-10d6ef1c-b777-4393-8e76-75eec29692bf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.879280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] The token cache is being invalidate because project 1af43682b4f7494bbcff28102e0f93e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:00.879685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.880392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:00.881999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1317/2656] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:00 2022] DELETE /identity/v3/projects/1af43682b4f7494bbcff28102e0f93e2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:00.886548 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:00.896546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aae026-42cb-443c-8ee8-31536eeb4df5 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:00.962890 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:00.964384 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:00.964835 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:00.965179 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] PATH_INFO: `/v3/users/44b0736cd3a64bea84718dc6c287739b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:00.966466 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b0736cd3a64bea84718dc6c287739b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:00.969085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.025345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b0736cd3a64bea84718dc6c287739b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.026001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b0736cd3a64bea84718dc6c287739b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.039201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b0736cd3a64bea84718dc6c287739b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.046398 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b0736cd3a64bea84718dc6c287739b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.055690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-84ce9cbf-f57c-4de3-a552-59906c8ba9b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b0736cd3a64bea84718dc6c287739b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.063332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1340/2657] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:00 2022] DELETE /identity/v3/users/44b0736cd3a64bea84718dc6c287739b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:01.348381 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.349962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:01.368257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6ef1c-b777-4393-8e76-75eec29692bf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.372176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:01.437802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.438459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.438739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.439013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] PATH_INFO: `/v3/projects/96c59edcc1c342e0924048ddc0bb5393` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.440344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c59edcc1c342e0924048ddc0bb5393)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.442283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.493268 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c59edcc1c342e0924048ddc0bb5393', 'request_id': 'req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.498966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c59edcc1c342e0924048ddc0bb5393', 'request_id': 'req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.506749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c59edcc1c342e0924048ddc0bb5393', 'request_id': 'req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.519183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] The token cache is being invalidate because project 96c59edcc1c342e0924048ddc0bb5393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:01.519502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.520164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c2b05d35-8d9a-4cd6-b142-3a89ba0c822f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:01.521706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1318/2658] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:01 2022] DELETE /identity/v3/projects/96c59edcc1c342e0924048ddc0bb5393 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:01.527356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89828f-80de-4762-a654-08a815611b24 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.527882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89828f-80de-4762-a654-08a815611b24 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.528006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89828f-80de-4762-a654-08a815611b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.547382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6c89828f-80de-4762-a654-08a815611b24 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:01.548573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6c89828f-80de-4762-a654-08a815611b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:01.592308 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:01.613715 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1341/2659] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:01.622140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.622435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.622716 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.643423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:01.644692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:01.653669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1319/2660] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:01.660814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89828f-80de-4762-a654-08a815611b24 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.672501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89828f-80de-4762-a654-08a815611b24 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.677729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.678394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.678670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.678940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.679672 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:28:01.679979 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.681570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.693294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1342/2661] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:28:01.697856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.708319 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71976846-7a51-4176-b17f-5976b91b9c33 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.712950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.713558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.714556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.714856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.715569 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.717510 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.745310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1320/2662] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:01 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:01.751025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.763817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a62c55-5951-484c-92cf-c1e0ca81b44a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.836287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.836918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.837201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.837476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.838142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.840335 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.866036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1343/2663] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:01 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:01.871785 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.892075 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1d97-66fc-4df9-b843-53b39278217d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.897186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.897865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.898150 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.898429 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.899143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.900831 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.911635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1321/2664] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:01.915768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.925754 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08114e-9be5-4723-a0c0-6079079768d0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.930266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f012cd-146d-47b4-814f-dfa1a5ad6621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.930872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f012cd-146d-47b4-814f-dfa1a5ad6621 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.931150 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f012cd-146d-47b4-814f-dfa1a5ad6621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.931421 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f012cd-146d-47b4-814f-dfa1a5ad6621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.932132 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f012cd-146d-47b4-814f-dfa1a5ad6621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.934476 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f012cd-146d-47b4-814f-dfa1a5ad6621 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:01.947071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1344/2665] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:01.952014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:01.962824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652af3da-69c5-41cc-a6e6-02a2438db839 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:01.968049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:01.968975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:01.969258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:01.969535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] PATH_INFO: `/v3/projects/b022f060e60c44318956a14e2babc56c/users/afb8f2cad9c848bb9c7579a359a73e44/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:01.984671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b022f060e60c44318956a14e2babc56c, user_id=afb8f2cad9c848bb9c7579a359a73e44, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:01.987304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.007814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1322/2666] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:01 2022] PUT /identity/v3/projects/b022f060e60c44318956a14e2babc56c/users/afb8f2cad9c848bb9c7579a359a73e44/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:02.016154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.016440 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.016711 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.046910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] MFA Rules not processed for user `afb8f2cad9c848bb9c7579a359a73e44`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:02.048173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:02.141274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1345/2667] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:02.142460 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.180131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:02.218217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64ed66-395f-4181-a298-75697bdb60be admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.223352 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.224018 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.224301 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.224579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.225274 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.227198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.266294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1323/2668] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:02 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:28:02.270333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.281735 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fd145-6ec4-4b9d-a3b9-4270bcba004f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.332429 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:02.364086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.365307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.365837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.366564 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.367541 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.370941 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.395958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1346/2669] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:02 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:02.401306 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.413426 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623ded9-d330-4e26-bd96-745f6c52edf6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.418684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.419304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.419621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.419893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.420590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.422176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.432395 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1324/2670] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:02.437254 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.447253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e047a1-f6f2-47f2-b4a6-cb35630fc6b3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.451902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.452560 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.452842 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.453115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] PATH_INFO: `/v3/projects/a6ff13da7dbc43279902efc9c52b0cba/users/658799ff59c9481e9a5db94d78348568/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.467703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ff13da7dbc43279902efc9c52b0cba, user_id=658799ff59c9481e9a5db94d78348568, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.471193 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.485490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1347/2671] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:02 2022] PUT /identity/v3/projects/a6ff13da7dbc43279902efc9c52b0cba/users/658799ff59c9481e9a5db94d78348568/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:02.489488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.500669 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff876ac1-cafc-46bf-91df-668ddbe89cfb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.556747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413562b-8e58-4117-8857-abe7f49c7b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.557361 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413562b-8e58-4117-8857-abe7f49c7b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.557740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413562b-8e58-4117-8857-abe7f49c7b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.558000 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413562b-8e58-4117-8857-abe7f49c7b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.558683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413562b-8e58-4117-8857-abe7f49c7b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.560283 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413562b-8e58-4117-8857-abe7f49c7b23 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.569950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1325/2672] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:02.574491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.584977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa50694-7920-4673-a47a-7b5e964427a6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.589895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.590552 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.590849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.591108 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] PATH_INFO: `/v3/projects/a6ff13da7dbc43279902efc9c52b0cba/users/658799ff59c9481e9a5db94d78348568/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.593060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ff13da7dbc43279902efc9c52b0cba, user_id=658799ff59c9481e9a5db94d78348568, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.595982 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.606880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1348/2673] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:02 2022] PUT /identity/v3/projects/a6ff13da7dbc43279902efc9c52b0cba/users/658799ff59c9481e9a5db94d78348568/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:02.618083 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.618379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.618663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.648118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] MFA Rules not processed for user `658799ff59c9481e9a5db94d78348568`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:02.649548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:02.729324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1326/2674] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:02.743324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.743324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:02.802381 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eab08d-0d29-4e75-a0dd-c44a5ec9002f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.806075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.806690 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.806973 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.807247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.816373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.817800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.827256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1349/2675] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:02.871357 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:02.872787 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:02.884894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bce7c-9787-477d-b3de-0b77d8c2945d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:02.888667 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac2303-9bdc-4f03-83b2-c220b28001ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:02.889305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac2303-9bdc-4f03-83b2-c220b28001ad service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:02.889589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac2303-9bdc-4f03-83b2-c220b28001ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:02.889896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac2303-9bdc-4f03-83b2-c220b28001ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:02.916499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:02.949144 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac2303-9bdc-4f03-83b2-c220b28001ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:02.950672 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac2303-9bdc-4f03-83b2-c220b28001ad service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:02.959862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1327/2676] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:03.349080 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:03.350490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:03.414411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b241ee2-0f05-44f7-a1e4-07c0db130ba2 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:03.418805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:03.419437 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:03.419745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:03.420025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] PATH_INFO: `/v3/users/306f2dcf3ec0498995b044113edbcb0c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:03.421178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306f2dcf3ec0498995b044113edbcb0c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:03.423078 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:03.463874 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:03.470094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306f2dcf3ec0498995b044113edbcb0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:03.471010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306f2dcf3ec0498995b044113edbcb0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:03.483355 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306f2dcf3ec0498995b044113edbcb0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:03.490577 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306f2dcf3ec0498995b044113edbcb0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:03.497917 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306f2dcf3ec0498995b044113edbcb0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:03.503872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1350/2677] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:03 2022] DELETE /identity/v3/users/306f2dcf3ec0498995b044113edbcb0c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:03.508442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:03.508940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:03.509053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:03.528346 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:03.529590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:03.587662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1328/2678] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:03.597830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:03.599256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:03.661519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9209e7-4a9d-4ee4-b051-6eb1d0e70e37 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:03.665318 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:03.665878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:03.666163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:03.666439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:03.676259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:03.677688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:03.687792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1351/2679] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:03.951929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:03.953401 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:03.965553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91b668-f821-4b2b-aa14-ff820527d9c3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:03.970484 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:03.971198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:03.971506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:03.971777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] PATH_INFO: `/v3/projects/ca5366bbbf864229979f8a9f936a1897` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:03.972975 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5366bbbf864229979f8a9f936a1897)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:03.974921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:04.025111 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5366bbbf864229979f8a9f936a1897', 'request_id': 'req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.031279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5366bbbf864229979f8a9f936a1897', 'request_id': 'req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.038982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5366bbbf864229979f8a9f936a1897', 'request_id': 'req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.050751 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] The token cache is being invalidate because project ca5366bbbf864229979f8a9f936a1897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:04.051070 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.051715 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.053210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1329/2680] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:03 2022] DELETE /identity/v3/projects/ca5366bbbf864229979f8a9f936a1897 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:04.057270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:04.067159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a08a1-8653-489c-bdcb-acb158754858 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:04.128501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:04.129141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.129426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:04.129704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] PATH_INFO: `/v3/users/9403ea7325d2406985fe344858baa7e0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:04.130862 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9403ea7325d2406985fe344858baa7e0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:04.132807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:04.179347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9403ea7325d2406985fe344858baa7e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.179964 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9403ea7325d2406985fe344858baa7e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.190202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:04.192669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9403ea7325d2406985fe344858baa7e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.200497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9403ea7325d2406985fe344858baa7e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.207673 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-20c8c894-d827-4bcc-a4bf-290c3c5d3bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9403ea7325d2406985fe344858baa7e0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.214482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1352/2681] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:04 2022] DELETE /identity/v3/users/9403ea7325d2406985fe344858baa7e0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:04.518131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:04.519732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:04.540588 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3feda-4cb6-44c6-9bb5-fcdf339ad537 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:04.594058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:04.608930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:04.609624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.609950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:04.610237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] PATH_INFO: `/v3/projects/cf15717a282045b2829b8de0d8fe4332` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:04.611431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf15717a282045b2829b8de0d8fe4332)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:04.613514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:04.675518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf15717a282045b2829b8de0d8fe4332', 'request_id': 'req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.682726 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf15717a282045b2829b8de0d8fe4332', 'request_id': 'req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.690566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf15717a282045b2829b8de0d8fe4332', 'request_id': 'req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.704514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] The token cache is being invalidate because project cf15717a282045b2829b8de0d8fe4332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:04.704827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.705469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b1b700ae-bfa7-44bf-9d8b-05d4f55e8d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:04.706962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1330/2682] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:04 2022] DELETE /identity/v3/projects/cf15717a282045b2829b8de0d8fe4332 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:04.711757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.712229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:04.712327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:04.735153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:04.736515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:04.804845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1353/2683] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:04.813702 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.814334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:04.814687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:04.835709 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:04.836942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:04.845296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1331/2684] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:04.849738 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:04.859327 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38795a-d354-4a45-8428-ff0c89bec04c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:04.864022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:04.864697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.864981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:04.865252 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:04.865940 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:28:04.866245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:04.867861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:04.879928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1354/2685] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:28:04.884369 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:04.894530 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52938bcc-7814-4679-982b-6b2b663bc7ff None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:04.899190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:04.899838 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.900151 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:04.900447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:04.901121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:04.903024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:04.929724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1332/2686] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:04 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:28:04.934554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:04.944401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef7de-e001-40a1-9b5f-7f8f7e095189 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:04.999028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:04.999662 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:04.999942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.000210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.000868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.002757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.024293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1355/2687] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:04 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:28:05.028451 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.038565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdacc60-2b4c-41a8-8dd7-66c079613029 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.043049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.043690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.043969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.044239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.044924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.046499 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.057053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1333/2688] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:05.061685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.071578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a0386-c146-4171-9a44-adcb84fc9f2e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.076998 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a4b98-87ac-4204-ac8b-238111de8b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.077612 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4b98-87ac-4204-ac8b-238111de8b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.077901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4b98-87ac-4204-ac8b-238111de8b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.078175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4b98-87ac-4204-ac8b-238111de8b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.078956 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4b98-87ac-4204-ac8b-238111de8b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.080525 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4b98-87ac-4204-ac8b-238111de8b8d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.091185 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1356/2689] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:28:05.095637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.105445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687a112-c531-48b5-bfc6-f792760b5bd4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.109994 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.110692 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.110927 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.111205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] PATH_INFO: `/v3/projects/5d643c28fd974799a7a9bec10aa7cadf/users/9fcb206f1dfb469ca2e7e53021baf6d1/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.127153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d643c28fd974799a7a9bec10aa7cadf, user_id=9fcb206f1dfb469ca2e7e53021baf6d1, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.129826 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.143086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1334/2690] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:05 2022] PUT /identity/v3/projects/5d643c28fd974799a7a9bec10aa7cadf/users/9fcb206f1dfb469ca2e7e53021baf6d1/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:05.210796 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.210991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.211256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.239154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] MFA Rules not processed for user `9fcb206f1dfb469ca2e7e53021baf6d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:05.240387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:05.272622 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:05.309741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1357/2691] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:05.314344 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.381118 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e5576-bbc4-4f83-9729-1941b9f3c0da admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.385883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.386498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.386783 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.387062 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.387786 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.389680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.418316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1335/2692] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:05 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:05.422359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.433610 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f81eb-80a1-407d-9fb5-cc7258c07887 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.489069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.489607 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.489881 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.490154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.490817 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.492784 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.516639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1358/2693] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:05 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:28:05.520926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.527311 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:05.534230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768dd13-ecab-4273-9df6-7c199e5ee910 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.539981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.540725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.541014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.541289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.541988 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.544174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.555243 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1336/2694] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:05.559750 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.570546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e018-6ae2-4eff-87da-a52fecfae9bb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.575094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.575788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.576070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.576336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] PATH_INFO: `/v3/projects/c8897fd3d4e841c29cc8c673ca48c1da/users/8b7de0df38a54a20bc3a36c93027d312/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.592485 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8897fd3d4e841c29cc8c673ca48c1da, user_id=8b7de0df38a54a20bc3a36c93027d312, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.595654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.609342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1359/2695] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:05 2022] PUT /identity/v3/projects/c8897fd3d4e841c29cc8c673ca48c1da/users/8b7de0df38a54a20bc3a36c93027d312/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:05.613632 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.624182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d8e66-33a9-4d6d-82f2-cf91ec746764 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.695879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d5e03-a418-4eb5-a625-7bc800cc6476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.696503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d5e03-a418-4eb5-a625-7bc800cc6476 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.696790 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d5e03-a418-4eb5-a625-7bc800cc6476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.697100 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d5e03-a418-4eb5-a625-7bc800cc6476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.697801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d5e03-a418-4eb5-a625-7bc800cc6476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.699442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d5e03-a418-4eb5-a625-7bc800cc6476 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.709905 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1337/2696] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:05.714853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.726215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2c63-fd9a-4635-9171-5cf8c886b875 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:05.731685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:05.732361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.732649 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.732932 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] PATH_INFO: `/v3/projects/c8897fd3d4e841c29cc8c673ca48c1da/users/8b7de0df38a54a20bc3a36c93027d312/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.734909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8897fd3d4e841c29cc8c673ca48c1da, user_id=8b7de0df38a54a20bc3a36c93027d312, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:05.737684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:05.753104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1360/2697] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:05 2022] PUT /identity/v3/projects/c8897fd3d4e841c29cc8c673ca48c1da/users/8b7de0df38a54a20bc3a36c93027d312/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:05.769255 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:05.769864 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:05.770449 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:05.808275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] MFA Rules not processed for user `8b7de0df38a54a20bc3a36c93027d312`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:05.809467 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:05.918488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1338/2698] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:05.931878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:05.934229 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:06.012188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd82476-6e17-405f-b5b3-08261cc7af7b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:06.015888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:06.016508 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.016885 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.017158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.027048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:06.028714 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:06.039913 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1361/2699] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:06.137791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:06.148133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4701b7-5bfe-4c48-ba1b-02ad4297a24c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:06.151814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:06.152470 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.152758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.153029 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.223419 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:06.224906 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:06.235678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1339/2700] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:06.453819 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:06.456016 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:06.506727 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:06.549984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7aa76-2bb1-4ec8-9109-31b67a45e964 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:06.557066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:06.558352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.558855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.559352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] PATH_INFO: `/v3/users/afb8f2cad9c848bb9c7579a359a73e44` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.560495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:06.561426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb8f2cad9c848bb9c7579a359a73e44)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:06.565271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:06.572007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0fd71-c5d8-40e6-8d7f-f9df169eae02 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:06.576738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:06.577457 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.577736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.578012 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] PATH_INFO: `/v3/users/9fcb206f1dfb469ca2e7e53021baf6d1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.579185 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fcb206f1dfb469ca2e7e53021baf6d1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:06.581396 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:06.588567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:06.621825 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb8f2cad9c848bb9c7579a359a73e44'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.622490 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb8f2cad9c848bb9c7579a359a73e44'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.635098 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb8f2cad9c848bb9c7579a359a73e44'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.642508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fcb206f1dfb469ca2e7e53021baf6d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.643827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fcb206f1dfb469ca2e7e53021baf6d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.647605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb8f2cad9c848bb9c7579a359a73e44'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.656692 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd5ff127-fdcc-410a-b780-127283c016af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb8f2cad9c848bb9c7579a359a73e44'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.663504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1362/2701] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:06 2022] DELETE /identity/v3/users/afb8f2cad9c848bb9c7579a359a73e44 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:06.669328 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.669635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.669950 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.676439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fcb206f1dfb469ca2e7e53021baf6d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.685279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fcb206f1dfb469ca2e7e53021baf6d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.686145 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-466080ed-b20a-40a6-b67d-921598a6e3b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fcb206f1dfb469ca2e7e53021baf6d1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:06.693930 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:06.695391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:06.703995 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1340/2702] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:06 2022] DELETE /identity/v3/users/9fcb206f1dfb469ca2e7e53021baf6d1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:06.708474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.708766 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.709049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.730570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:06.731848 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:06.759351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1363/2703] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:06.769921 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:06.772451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:06.811601 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1341/2704] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:06.821140 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:06.822701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:06.834077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:06.839824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903a3a1-7fb9-4c7f-8f6f-69b73bff2360 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:06.892807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c7ecd-451e-4f9c-aafb-87207b569aeb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:06.896546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:06.897338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.897625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.897928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.898336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:06.899077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:06.899358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:06.899665 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:06.908959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:06.910235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:06.910407 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:06.911687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:06.922344 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1364/2705] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:28:06.923195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1342/2706] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:28:07.322926 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:07.325064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:07.338643 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69691c4a-8092-4600-872d-79846976f330 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:07.358558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:07.359413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:07.359809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:07.360211 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] PATH_INFO: `/v3/projects/b022f060e60c44318956a14e2babc56c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:07.362126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b022f060e60c44318956a14e2babc56c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:07.364895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:07.426680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b022f060e60c44318956a14e2babc56c', 'request_id': 'req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.437938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b022f060e60c44318956a14e2babc56c', 'request_id': 'req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.447299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b022f060e60c44318956a14e2babc56c', 'request_id': 'req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.460515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] The token cache is being invalidate because project b022f060e60c44318956a14e2babc56c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:07.460838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.461513 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.463329 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1365/2707] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:07 2022] DELETE /identity/v3/projects/b022f060e60c44318956a14e2babc56c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:07.468544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:07.479652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcd30-7ea8-4d1a-ab63-d30d49d5f966 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:07.508759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:07.561606 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:07.563123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:07.564104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:07.564393 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] PATH_INFO: `/v3/users/658799ff59c9481e9a5db94d78348568` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:07.565782 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] RBAC: Authorizing `identity:delete_user(user_id=658799ff59c9481e9a5db94d78348568)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:07.567721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:07.632228 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658799ff59c9481e9a5db94d78348568'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.633211 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658799ff59c9481e9a5db94d78348568'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.647243 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:07.650128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:07.677014 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658799ff59c9481e9a5db94d78348568'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.677339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cc3e5-fe14-4dc9-933f-8f59da90eafe admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:07.685062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:07.685849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:07.686202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:07.686558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] PATH_INFO: `/v3/projects/5d643c28fd974799a7a9bec10aa7cadf` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:07.688076 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658799ff59c9481e9a5db94d78348568'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.688228 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d643c28fd974799a7a9bec10aa7cadf)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:07.688899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2095bc6e-1247-471f-8705-ce4b8ca5d50f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658799ff59c9481e9a5db94d78348568'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.691233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:07.705362 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1343/2708] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:07 2022] DELETE /identity/v3/users/658799ff59c9481e9a5db94d78348568 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:07.706869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:07.707161 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:07.707609 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:07.733123 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] MFA Rules not processed for user `540bec8413b144889a07e9d9458b298e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:07.734466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:07.754037 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d643c28fd974799a7a9bec10aa7cadf', 'request_id': 'req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.764053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d643c28fd974799a7a9bec10aa7cadf', 'request_id': 'req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.772679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d643c28fd974799a7a9bec10aa7cadf', 'request_id': 'req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.788265 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] The token cache is being invalidate because project 5d643c28fd974799a7a9bec10aa7cadf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:07.788625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.789330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.791006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1366/2709] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:07 2022] DELETE /identity/v3/projects/5d643c28fd974799a7a9bec10aa7cadf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:07.796173 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:07.797653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:07.816358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1344/2710] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:07.817287 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:07.909346 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65f5bb-1027-44f7-9dd3-b38448ba37d4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:07.920300 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:07.921624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:07.922143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:07.922776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] PATH_INFO: `/v3/users/8b7de0df38a54a20bc3a36c93027d312` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:07.925394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7de0df38a54a20bc3a36c93027d312)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:07.931270 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:07.998370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7de0df38a54a20bc3a36c93027d312'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:07.999164 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7de0df38a54a20bc3a36c93027d312'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.012671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7de0df38a54a20bc3a36c93027d312'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.020742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7de0df38a54a20bc3a36c93027d312'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.028709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-edfdda25-0d9b-4370-a346-779de0b50da0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7de0df38a54a20bc3a36c93027d312'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.035420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1367/2711] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:07 2022] DELETE /identity/v3/users/8b7de0df38a54a20bc3a36c93027d312 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:08.129282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.131885 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.152962 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09232132-89b5-46d6-a0eb-007aa54db034 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.243157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:08.262125 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.263554 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.263960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.264453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] PATH_INFO: `/v3/projects/a6ff13da7dbc43279902efc9c52b0cba` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.266342 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ff13da7dbc43279902efc9c52b0cba)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.268546 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.331579 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ff13da7dbc43279902efc9c52b0cba', 'request_id': 'req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.339807 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ff13da7dbc43279902efc9c52b0cba', 'request_id': 'req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.352154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ff13da7dbc43279902efc9c52b0cba', 'request_id': 'req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.372022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] The token cache is being invalidate because project a6ff13da7dbc43279902efc9c52b0cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:08.372638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.373492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.375400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1345/2712] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:08 2022] DELETE /identity/v3/projects/a6ff13da7dbc43279902efc9c52b0cba => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:08.384058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d37e4c-4ea5-4cee-a1cb-e722824844b4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.384668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d37e4c-4ea5-4cee-a1cb-e722824844b4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.385356 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d37e4c-4ea5-4cee-a1cb-e722824844b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.401648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.404858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.410205 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-07d37e4c-4ea5-4cee-a1cb-e722824844b4 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:08.412245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-07d37e4c-4ea5-4cee-a1cb-e722824844b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.488338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2d567-c0c6-4a53-b045-4d2c1c88137c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.494315 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.494967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.495289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.495597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] PATH_INFO: `/v3/projects/c8897fd3d4e841c29cc8c673ca48c1da` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.497010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8897fd3d4e841c29cc8c673ca48c1da)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.498987 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.503510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1368/2713] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:08.507740 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.508089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.508491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.533074 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:08.534726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.547836 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1369/2714] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:08.552221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.562497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd507078-4554-4625-9fca-c88f5e7cd45d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.566056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8897fd3d4e841c29cc8c673ca48c1da', 'request_id': 'req-93f3ff15-81b0-45d4-a507-f58c88eeeb96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.572661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8897fd3d4e841c29cc8c673ca48c1da', 'request_id': 'req-93f3ff15-81b0-45d4-a507-f58c88eeeb96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.583772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8897fd3d4e841c29cc8c673ca48c1da', 'request_id': 'req-93f3ff15-81b0-45d4-a507-f58c88eeeb96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.591905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:08.600812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] The token cache is being invalidate because project c8897fd3d4e841c29cc8c673ca48c1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:08.601299 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.601925 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93f3ff15-81b0-45d4-a507-f58c88eeeb96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:08.604073 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1346/2715] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:08 2022] DELETE /identity/v3/projects/c8897fd3d4e841c29cc8c673ca48c1da => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:08.609037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934636f-d455-4388-a035-816630e3a699 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.609331 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934636f-d455-4388-a035-816630e3a699 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.609618 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934636f-d455-4388-a035-816630e3a699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.633047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0934636f-d455-4388-a035-816630e3a699 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:08.633591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.634339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0934636f-d455-4388-a035-816630e3a699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.634587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.634931 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.635299 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.636702 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:28:08.637077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.638782 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.647523 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1347/2716] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:08.655662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.655937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.656217 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.667083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1370/2717] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:28:08.668258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:08.672665 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.674977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.684175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:08.688127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:08.690801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3d25-dee6-418d-ac8c-00586db00249 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.696396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.697023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.697307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.697593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.697804 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1348/2718] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:08.698882 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.700847 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.702686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.716957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40109a-694c-45f2-8fef-625483537d4c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.724977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.726131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.726686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.727415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.728456 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:28:08.728964 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.731897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.738676 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1371/2719] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:28:08.742889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.752184 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1349/2720] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:28:08.757704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.770069 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3598421-f6f1-46b5-bbde-d8ab27ad615a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.830041 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73c0e-5616-47bf-9afd-921ac80e4869 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.835194 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.835928 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.836310 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.836920 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.837776 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.840156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.857183 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.857948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.858289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.858648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.859493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.861624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.868743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1372/2721] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:08.869710 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.880909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ecd2-ae92-4fab-87b5-f3b2a7f3b9fa admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.891072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.894534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.894814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.895096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.895895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.898336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:08.899996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1350/2722] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:28:08.901293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.921923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1373/2723] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:28:08.925995 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:08.939049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f5d1-78b6-4bd5-8c9c-9d8b90213592 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.978022 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de46b7-6a7f-4088-827b-ac3c6fb8784a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:08.984039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:08.984681 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:08.984966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:08.985247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:08.985932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:08.987897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.014430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1351/2724] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:08 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:09.015496 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:09.019097 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.019823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.020210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.020491 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.021221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.022970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.026957 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e576c31-5e08-4a62-8708-f038af633ceb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:09.034151 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1374/2725] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:28:09.034916 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.035594 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.035868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.036149 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.036856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.038617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.039088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:09.051727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1352/2726] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:09.055797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:09.055936 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1258f-19f0-4460-94f4-14d6a1760d56 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:09.063664 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522ce12-43bf-47f2-a674-56dace567f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.064631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ce12-43bf-47f2-a674-56dace567f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.065124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ce12-43bf-47f2-a674-56dace567f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.065593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ce12-43bf-47f2-a674-56dace567f40 admin admin] PATH_INFO: `/v3/projects/2a6eb59bc9a445cc9f5b4d3059fbb7f1/users/6040773eefe146a0adbc7858c7371cb8/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.067154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915e49-ba37-4a6a-bd56-3e1191245c17 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:09.074236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.074965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.075250 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.075563 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.076339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.078358 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.083636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:09.087613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522ce12-43bf-47f2-a674-56dace567f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6eb59bc9a445cc9f5b4d3059fbb7f1, user_id=6040773eefe146a0adbc7858c7371cb8, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.091729 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522ce12-43bf-47f2-a674-56dace567f40 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.099647 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1353/2727] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:28:09.104209 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:09.107650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1375/2728] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:09 2022] PUT /identity/v3/projects/2a6eb59bc9a445cc9f5b4d3059fbb7f1/users/6040773eefe146a0adbc7858c7371cb8/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:09.112743 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.113154 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.113506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.116464 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f85d9-4547-4c75-8225-25e9a6a485ff admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:09.133075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:09.134387 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:09.157222 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:09.188027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772732d-a158-40ae-bcd2-25e959a8ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.188758 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772732d-a158-40ae-bcd2-25e959a8ecb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.189270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772732d-a158-40ae-bcd2-25e959a8ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.189388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772732d-a158-40ae-bcd2-25e959a8ecb1 admin admin] PATH_INFO: `/v3/projects/ebd1f44d0fb34cad9e350f51f23ce6f7/users/7d6c937ec12e4397adb4d0bc3ae2bc77/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.205568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772732d-a158-40ae-bcd2-25e959a8ecb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd1f44d0fb34cad9e350f51f23ce6f7, user_id=7d6c937ec12e4397adb4d0bc3ae2bc77, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.207824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1376/2729] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:09.208360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772732d-a158-40ae-bcd2-25e959a8ecb1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.218315 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:09.220200 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:09.224577 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1354/2730] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:09 2022] PUT /identity/v3/projects/ebd1f44d0fb34cad9e350f51f23ce6f7/users/7d6c937ec12e4397adb4d0bc3ae2bc77/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:09.229463 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.229781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.230047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.249164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:09.250392 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:09.312063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e319ab-8a9e-475f-b2bc-ce11074647f9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:09.320648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1355/2731] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:28:09.326701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:09.328215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:09.381879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81c3f2-56eb-4ab7-8218-f076703f2e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.382522 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81c3f2-56eb-4ab7-8218-f076703f2e4b service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.382818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81c3f2-56eb-4ab7-8218-f076703f2e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.383082 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81c3f2-56eb-4ab7-8218-f076703f2e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.397048 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81c3f2-56eb-4ab7-8218-f076703f2e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.398507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81c3f2-56eb-4ab7-8218-f076703f2e4b service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.410010 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56272-4321-46c3-86a6-9c241286409e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:09.411133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1377/2732] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:28:09.420778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:09.421929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:09.422242 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:09.422519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:09.435503 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:09.437013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:09.450273 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1356/2733] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:10.490745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:10.492759 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:10.493206 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:10.525562 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] MFA Rules not processed for user `7d6c937ec12e4397adb4d0bc3ae2bc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:10.527742 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:10.577184 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:10.624010 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1378/2734] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:10.629418 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:10.645912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b2d2b-1ba0-4146-b0c5-73608d159404 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:10.646893 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:10.647583 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:10.647867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:10.648143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:10.648967 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:10.651272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:10.692811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:10.702980 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1357/2735] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:10 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:10.706020 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42cf1b-d823-4abd-94c6-2f2017ad656f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:10.801962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:10.802866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:10.803433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:10.803854 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:10.804880 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:10.808431 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:10.837935 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1379/2736] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:10 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:10.840199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:10.854174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d277ae-2bd1-4c20-a5b5-f4fb3d6eb978 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:10.862188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:10.863291 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:10.863760 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:10.864208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:10.865362 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:10.867810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:10.885671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1358/2737] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:10.887663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:10.899197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d495a2f-e26a-48df-a0d6-61ad4ea8c7de admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:10.906363 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:10.907962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:10.908374 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:10.908838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] PATH_INFO: `/v3/projects/4affd20383bd47938f16b345cc76f524/users/619643506ce9437d883a3329a50d415d/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:10.936351 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4affd20383bd47938f16b345cc76f524, user_id=619643506ce9437d883a3329a50d415d, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:10.941628 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:10.969991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1380/2738] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:10 2022] PUT /identity/v3/projects/4affd20383bd47938f16b345cc76f524/users/619643506ce9437d883a3329a50d415d/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:10.975996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:11.025580 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2719e96-b629-4523-9752-5eb8c079ed89 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:11.096270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:11.140023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc9b66-dfca-40ed-adf2-619a10efdcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:11.140765 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc9b66-dfca-40ed-adf2-619a10efdcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:11.141199 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc9b66-dfca-40ed-adf2-619a10efdcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:11.141575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc9b66-dfca-40ed-adf2-619a10efdcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:11.142461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc9b66-dfca-40ed-adf2-619a10efdcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:11.145519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc9b66-dfca-40ed-adf2-619a10efdcf2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:11.163879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1359/2739] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:11.165186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:11.178129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f8568-f8b2-4d05-99d5-464c20fcce37 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:11.189753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:11.193855 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:11.194093 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:11.194897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] PATH_INFO: `/v3/projects/4affd20383bd47938f16b345cc76f524/users/619643506ce9437d883a3329a50d415d/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:11.205220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4affd20383bd47938f16b345cc76f524, user_id=619643506ce9437d883a3329a50d415d, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:11.215749 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:11.267589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1381/2740] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:11 2022] PUT /identity/v3/projects/4affd20383bd47938f16b345cc76f524/users/619643506ce9437d883a3329a50d415d/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:12.128857 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:12.129400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:12.129400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:12.161582 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] MFA Rules not processed for user `619643506ce9437d883a3329a50d415d`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:12.162840 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:12.268711 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1360/2741] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:12.274117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:12.275910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:12.358046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d15c8-b7ae-4178-be8e-16c446436252 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:12.363372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad62641-97f0-4008-8fae-643219011648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:12.364271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad62641-97f0-4008-8fae-643219011648 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:12.364650 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad62641-97f0-4008-8fae-643219011648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:12.365104 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad62641-97f0-4008-8fae-643219011648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:12.439557 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:12.468221 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad62641-97f0-4008-8fae-643219011648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:12.471546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad62641-97f0-4008-8fae-643219011648 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:12.486195 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1382/2742] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:14.130735 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:14.141436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc250c4-86fc-462e-aaee-1c8e75454b5e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:14.154437 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:14.155099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:14.155481 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:14.155691 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:14.167064 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:14.169088 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:14.180558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1361/2743] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:16.178942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.179534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.179534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.209289 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] MFA Rules not processed for user `6040773eefe146a0adbc7858c7371cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:16.210527 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:16.280519 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1383/2744] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:16.285478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:16.359175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b276ab-ea2c-4a3a-a6c4-981ad9b05fed service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:16.363755 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:16.364370 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.364654 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.364926 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.365608 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:16.367516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:16.393495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1362/2745] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:28:16 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:16.397805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:16.409188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c365110-f229-4c1d-ace2-db499e2df6b7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:16.466235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:16.466871 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.467149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.467420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.468122 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:16.470319 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:16.492757 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1384/2746] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:28:16 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:28:16.497173 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:16.510544 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f39a2-48b4-451c-9e65-19bcb4528723 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:16.514953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:16.515630 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.515922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.516205 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.516898 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:16.518555 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:16.531011 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1363/2747] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:16.535497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:16.546000 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7f9f4-175f-47d5-9c11-eece5213bc62 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:16.550775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:16.551441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.551745 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.552019 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] PATH_INFO: `/v3/projects/822f27f0d71f46809595eba71e693250/users/5f361d19b28c49e8b406bd0760a08633/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.566959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822f27f0d71f46809595eba71e693250, user_id=5f361d19b28c49e8b406bd0760a08633, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:16.569686 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:16.584007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1385/2748] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:16 2022] PUT /identity/v3/projects/822f27f0d71f46809595eba71e693250/users/5f361d19b28c49e8b406bd0760a08633/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:16.588071 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:16.599502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d024-5253-47a8-9fad-819dc15c7407 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:16.608999 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:16.665155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81364d70-8d64-490a-979d-40c82041a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:16.665796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81364d70-8d64-490a-979d-40c82041a62a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.666078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81364d70-8d64-490a-979d-40c82041a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.666351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81364d70-8d64-490a-979d-40c82041a62a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.667230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81364d70-8d64-490a-979d-40c82041a62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:16.669175 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81364d70-8d64-490a-979d-40c82041a62a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:16.682868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1364/2749] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:28:16.684598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:16.695046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e36c7-9168-45fd-8161-c7f205e27512 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:16.699606 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:16.700260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:16.700539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:16.700814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] PATH_INFO: `/v3/projects/822f27f0d71f46809595eba71e693250/users/5f361d19b28c49e8b406bd0760a08633/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:16.702755 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822f27f0d71f46809595eba71e693250, user_id=5f361d19b28c49e8b406bd0760a08633, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:16.705404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:16.715556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1386/2750] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:28:16 2022] PUT /identity/v3/projects/822f27f0d71f46809595eba71e693250/users/5f361d19b28c49e8b406bd0760a08633/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:21.925725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796cde-3bcd-4ba6-a5ea-c8897cd73c28 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:21.926321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796cde-3bcd-4ba6-a5ea-c8897cd73c28 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:21.926321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796cde-3bcd-4ba6-a5ea-c8897cd73c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:21.958239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4b796cde-3bcd-4ba6-a5ea-c8897cd73c28 None None] MFA Rules not processed for user `5f361d19b28c49e8b406bd0760a08633`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:21.959732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4b796cde-3bcd-4ba6-a5ea-c8897cd73c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:22.093966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1365/2751] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:22.196981 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:22.199372 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:22.243096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:22.376475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e5efd-b36a-423f-9bf1-63e6998a2d23 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:22.382578 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:22.383410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:22.383839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:22.384218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:22.475370 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:22.477361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:22.488573 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1387/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:28:23.830558 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:23.831371 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:23.831708 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:23.851933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] MFA Rules not processed for user `6040773eefe146a0adbc7858c7371cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:28:23.853152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:23.860881 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1366/2753] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:28:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:28:27.348204 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:27.410653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca69e22-e37b-4026-8e18-eb9df832bcff service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:27.417036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:27.417718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:27.417999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:27.418271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:27.420111 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:27.481685 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:27.504866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:27.506977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:27.527816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1388/2754] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:29.354461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:29.427372 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05121a8d-2a17-47cc-a1a8-657f3b13ce6e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:29.432290 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:29.432937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:29.433221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:29.433600 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] PATH_INFO: `/v3/users/7d6c937ec12e4397adb4d0bc3ae2bc77` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:29.434795 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6c937ec12e4397adb4d0bc3ae2bc77)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:29.436756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:29.481896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:29.503091 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6c937ec12e4397adb4d0bc3ae2bc77'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:29.504046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6c937ec12e4397adb4d0bc3ae2bc77'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:29.521924 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6c937ec12e4397adb4d0bc3ae2bc77'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:29.529801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6c937ec12e4397adb4d0bc3ae2bc77'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:29.530522 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6c937ec12e4397adb4d0bc3ae2bc77'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:29.544990 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1367/2755] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:29 2022] DELETE /identity/v3/users/7d6c937ec12e4397adb4d0bc3ae2bc77 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:29.931166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:29.943534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc13553-8bbd-4e4e-b51a-a130f6ade890 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:30.019141 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:30.024663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:30.025143 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:30.025288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] PATH_INFO: `/v3/projects/ebd1f44d0fb34cad9e350f51f23ce6f7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:30.026668 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd1f44d0fb34cad9e350f51f23ce6f7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:30.029422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:30.088532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd1f44d0fb34cad9e350f51f23ce6f7', 'request_id': 'req-e650b163-a608-453d-a4b3-405ab4b6d700'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.096269 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd1f44d0fb34cad9e350f51f23ce6f7', 'request_id': 'req-e650b163-a608-453d-a4b3-405ab4b6d700'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.107320 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd1f44d0fb34cad9e350f51f23ce6f7', 'request_id': 'req-e650b163-a608-453d-a4b3-405ab4b6d700'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.124461 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] The token cache is being invalidate because project ebd1f44d0fb34cad9e350f51f23ce6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:30.124756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.125480 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.127306 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1389/2756] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:29 2022] DELETE /identity/v3/projects/ebd1f44d0fb34cad9e350f51f23ce6f7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:30.131479 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:30.133053 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:30.146566 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f6c2-d6d7-4b96-a7c2-fd654225805b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:30.224195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:30.224879 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:30.225163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:30.225439 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] PATH_INFO: `/v3/users/619643506ce9437d883a3329a50d415d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:30.226693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=619643506ce9437d883a3329a50d415d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:30.228774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:30.303935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619643506ce9437d883a3329a50d415d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.305246 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619643506ce9437d883a3329a50d415d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.344455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619643506ce9437d883a3329a50d415d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.355305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619643506ce9437d883a3329a50d415d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.356469 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619643506ce9437d883a3329a50d415d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.376818 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1368/2757] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:28:30 2022] DELETE /identity/v3/users/619643506ce9437d883a3329a50d415d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:28:30.619183 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:30.626298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:30.642031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650b163-a608-453d-a4b3-405ab4b6d700 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:30.737644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:30.738626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:30.738905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:30.739177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] PATH_INFO: `/v3/projects/4affd20383bd47938f16b345cc76f524` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:30.740389 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=4affd20383bd47938f16b345cc76f524)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:30.742324 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:30.765160 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:30.796443 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4affd20383bd47938f16b345cc76f524', 'request_id': 'req-31441da6-0448-4582-a61a-60b99edb12ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.807451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4affd20383bd47938f16b345cc76f524', 'request_id': 'req-31441da6-0448-4582-a61a-60b99edb12ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.819103 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4affd20383bd47938f16b345cc76f524', 'request_id': 'req-31441da6-0448-4582-a61a-60b99edb12ec'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.832115 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] The token cache is being invalidate because project 4affd20383bd47938f16b345cc76f524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:28:30.832493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.833149 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:28:30.834687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1390/2758] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:28:30 2022] DELETE /identity/v3/projects/4affd20383bd47938f16b345cc76f524 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:28:39.496128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:39.498388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:39.533914 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:39.566262 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58c2d6-4beb-4f5f-9a58-ce7d5897b3d3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:39.623035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:39.623689 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:39.623970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:39.624327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:39.626367 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:39.636516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:39.637959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:39.650008 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1369/2759] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:40.617790 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:40.619359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:40.714247 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31441da6-0448-4582-a61a-60b99edb12ec admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:40.719364 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:40.720008 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:40.720288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:40.720589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:40.722778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:40.734007 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:40.735574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:40.745190 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1391/2760] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:28:49.643803 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=34 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:50.741059 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:56.945468 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:56.947188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:56.958215 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc9e56-9f05-409d-a023-b85771b93b32 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:56.961810 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:56.962412 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:56.962690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:56.962958 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:56.964656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:57.027067 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:57.028531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64638a43-183f-4331-8358-64ed881e5ef6 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:57.037930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1370/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:29:07.033764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=30 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:30:12.931587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:30:12.933033 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:12.947523 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8376-e4ea-4fd4-bce0-ac4e2ae42169 service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:30:12.951177 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:30:12.951867 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:30:12.952261 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:30:12.953046 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:30:12.954899 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:13.073279 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:30:13.074766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:30:13.089175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1392/2762] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:30:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:30:17.387058 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbfc86-345e-4391-9d48-74f43d20d877 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:30:17.387749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbfc86-345e-4391-9d48-74f43d20d877 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:30:17.387749 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbfc86-345e-4391-9d48-74f43d20d877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:30:17.408085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5fbbfc86-345e-4391-9d48-74f43d20d877 None None] MFA Rules not processed for user `4079529de70f4994ab9f1f3dd5cb3fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:30:17.409332 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5fbbfc86-345e-4391-9d48-74f43d20d877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:17.474894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1371/2763] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:30:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:30:18.226728 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:30:18.240415 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e86a1-8f4a-4017-8284-392bc06d5a7f service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:30:18.244373 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:30:18.245005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:30:18.245303 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:30:18.245911 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:30:18.247679 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:18.268360 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:30:18.270076 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:30:18.283660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1393/2764] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:30:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:30:27.465500 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=25 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:30:28.276800 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=40 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:30:32.749188 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:30:32.749764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:30:32.749764 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:30:32.771190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] MFA Rules not processed for user `540bec8413b144889a07e9d9458b298e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:30:32.772504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:32.857723 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1372/2765] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:30:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:30:36.074517 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:30:36.090259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942df660-26ff-4a7e-a020-66952743c101 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:30:36.099159 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:30:36.100217 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:30:36.101681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:30:36.102903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:30:36.106938 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:36.124539 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:30:36.126824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:30:36.142144 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1394/2766] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:30:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:30:42.849128 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=25 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:30:46.133555 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:30:54.015695 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:30:54.017259 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:54.041584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032c425-5b5e-4c3d-a17b-66929037c222 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:30:54.047204 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a432267-2873-46fc-b138-92f412838607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:30:54.048422 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a432267-2873-46fc-b138-92f412838607 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:30:54.048704 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a432267-2873-46fc-b138-92f412838607 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:30:54.048976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a432267-2873-46fc-b138-92f412838607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:30:54.051397 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5a432267-2873-46fc-b138-92f412838607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:30:54.064603 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a432267-2873-46fc-b138-92f412838607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:30:54.066241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a432267-2873-46fc-b138-92f412838607 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:30:54.078745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1373/2767] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:04.073688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:11.069126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:11.078214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b273c8d-29cd-46df-9ec2-822eb48159a9 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:11.082277 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:11.082980 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:11.083325 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:11.083653 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:11.085546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:11.096418 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:11.097865 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:11.107646 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1395/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:11.215258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a432267-2873-46fc-b138-92f412838607 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:11.226007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a432267-2873-46fc-b138-92f412838607 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:11.229930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:11.230531 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:11.230815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:11.231085 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:11.232897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:11.246865 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:11.248339 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:11.260493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1374/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:11.324465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:11.337896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab44825-dbcb-42db-b9a2-3648d6e2b5fb service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:11.343146 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:11.343829 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:11.344110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:11.344383 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:11.346901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:11.359481 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:11.361070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:11.371106 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1396/2770] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:13.044662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:13.058236 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44354e-67e8-48ae-82b6-a81bc003b81b service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:13.064455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:13.065077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:13.065492 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:13.065774 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:13.067573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:13.081231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:13.082968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:13.095763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1375/2771] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:21.366774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=12 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:23.092133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=12 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:34.828439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:34.830581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:34.847954 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367706-394b-45e4-9f3a-54e72f5ccdba service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:34.854023 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b5f371cd614e3bb246b015818a97f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:34.855065 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:34.855451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:34.855942 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:34.858072 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:34.872051 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:34.874414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:34.890433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1397/2772] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:31:42.892912 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:42.894652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:42.964922 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9537a-777d-4e45-a146-87b175329759 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:42.969310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:42.969941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:42.970276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:42.970552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] PATH_INFO: `/v3/users/4079529de70f4994ab9f1f3dd5cb3fd7` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:42.971747 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4079529de70f4994ab9f1f3dd5cb3fd7)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:42.973649 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.020701 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4079529de70f4994ab9f1f3dd5cb3fd7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.021408 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4079529de70f4994ab9f1f3dd5cb3fd7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.035090 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4079529de70f4994ab9f1f3dd5cb3fd7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.042920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4079529de70f4994ab9f1f3dd5cb3fd7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.043687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-984f6f42-3068-4eac-bbd3-56459dd0e303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4079529de70f4994ab9f1f3dd5cb3fd7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.063798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1376/2773] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:31:42 2022] DELETE /identity/v3/users/4079529de70f4994ab9f1f3dd5cb3fd7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:43.255096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.265904 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431e32d-154c-404f-ae8b-8abf2536c347 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.322985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.323680 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.323949 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.324227 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] PATH_INFO: `/v3/projects/00a93cc41a184683a1fb82acfa2660a6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.325536 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a93cc41a184683a1fb82acfa2660a6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.327441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.374774 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a93cc41a184683a1fb82acfa2660a6', 'request_id': 'req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.380404 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a93cc41a184683a1fb82acfa2660a6', 'request_id': 'req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.388707 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a93cc41a184683a1fb82acfa2660a6', 'request_id': 'req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.404739 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] The token cache is being invalidate because project 00a93cc41a184683a1fb82acfa2660a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:31:43.405028 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.405688 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0f484a2a-0c76-4eaa-81dd-8ea4089566a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:43.407202 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1398/2774] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:31:43 2022] DELETE /identity/v3/projects/00a93cc41a184683a1fb82acfa2660a6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:43.413163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.413459 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.413741 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.433856 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:43.435078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:43.495244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1377/2775] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:43.503188 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.503411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.503732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.523424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:43.524648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:43.532400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1399/2776] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:43.537127 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.547006 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fb11-b030-49c2-810b-6acce12a7f33 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.551693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.552318 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.552602 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.552877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.553590 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:31:43.553899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.555532 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.567584 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1378/2777] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:31:43.572267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.582210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f6764-329d-487d-a2f8-c56cae909a00 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.587094 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.587747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.588032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.588307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.588969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.590853 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.619192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1400/2778] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:31:43 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:31:43.623416 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.633254 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef142e-f535-4ac8-9697-2c104bdec61f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.636732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:43.692136 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.692778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.693033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.693303 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.693965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.695896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.717685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1379/2779] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:31:43 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:31:43.722592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.733136 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c1d7-7cd3-4cc9-b1ce-ca49f47a3819 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.737723 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.738331 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.738613 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.738888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.739647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.741223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.751069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1401/2780] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:43.755208 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.765181 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7ca18-e766-46ea-a741-29cbee767ab5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.770338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c33f06-15aa-4543-8c05-da4e9c7c44f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.770936 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c33f06-15aa-4543-8c05-da4e9c7c44f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.771219 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c33f06-15aa-4543-8c05-da4e9c7c44f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.771525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c33f06-15aa-4543-8c05-da4e9c7c44f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.772237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c33f06-15aa-4543-8c05-da4e9c7c44f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.773823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c33f06-15aa-4543-8c05-da4e9c7c44f3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.785013 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1380/2781] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:43.789045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:43.799002 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5fa0c-b42d-4560-a5ec-8f6ca852604c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:43.803629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:43.804288 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.804570 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.804844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] PATH_INFO: `/v3/projects/8185917982a14e1f85285ee8da832473/users/5380dc2313c142dd952407c5cc74b608/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.824447 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8185917982a14e1f85285ee8da832473, user_id=5380dc2313c142dd952407c5cc74b608, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:43.827088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:43.841411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1402/2782] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:43 2022] PUT /identity/v3/projects/8185917982a14e1f85285ee8da832473/users/5380dc2313c142dd952407c5cc74b608/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:43.905526 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:43.906107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:43.906107 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:43.934376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] MFA Rules not processed for user `5380dc2313c142dd952407c5cc74b608`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:43.935635 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:43.999186 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1381/2783] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:44.003809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.017313 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:44.067470 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d278e-893e-4b32-95cd-e10ebbeae457 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.072216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.072837 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.073118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.073391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.074066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.075985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.102556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1403/2784] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:31:44 2022] POST /identity/v3/projects => generated 361 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:31:44.106972 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.118493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5394-23b5-48d5-8a88-4603bc327830 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.175674 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.176295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.176572 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.176842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.177512 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.179409 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.199732 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1382/2785] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:31:44 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:31:44.203780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.214751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b00b5-abb9-4677-b4db-523b643582b6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.219635 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.220801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.221083 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.221492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.222197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.223844 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.235451 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1404/2786] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:44.239968 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.246501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:44.250644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43733b-e8bb-4831-a388-d8ea90fc0b0f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.256351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.257035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.257316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.257592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] PATH_INFO: `/v3/projects/5e614bd4d8f14a1cb4166f2d23babc13/users/9cb8e7b3b73947ab92be957f5b4ec5e8/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.273663 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e614bd4d8f14a1cb4166f2d23babc13, user_id=9cb8e7b3b73947ab92be957f5b4ec5e8, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.276266 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.290835 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1383/2787] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:44 2022] PUT /identity/v3/projects/5e614bd4d8f14a1cb4166f2d23babc13/users/9cb8e7b3b73947ab92be957f5b4ec5e8/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:44.294944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.305830 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ebbc1-559b-4747-97f1-8afce65c3e0c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.363147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47881b86-140c-42ae-9da7-3f0ec99bb2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.363805 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47881b86-140c-42ae-9da7-3f0ec99bb2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.364089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47881b86-140c-42ae-9da7-3f0ec99bb2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.364365 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47881b86-140c-42ae-9da7-3f0ec99bb2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.365060 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47881b86-140c-42ae-9da7-3f0ec99bb2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.366654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47881b86-140c-42ae-9da7-3f0ec99bb2d1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.376497 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1405/2788] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:31:44.380687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.390933 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb219b-4ea9-4336-af7d-26c46c250d62 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.396087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.396750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.397026 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.397304 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] PATH_INFO: `/v3/projects/5e614bd4d8f14a1cb4166f2d23babc13/users/9cb8e7b3b73947ab92be957f5b4ec5e8/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.399289 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e614bd4d8f14a1cb4166f2d23babc13, user_id=9cb8e7b3b73947ab92be957f5b4ec5e8, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.401974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.412593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1384/2789] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:44 2022] PUT /identity/v3/projects/5e614bd4d8f14a1cb4166f2d23babc13/users/9cb8e7b3b73947ab92be957f5b4ec5e8/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:44.422192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.422482 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.422765 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.452345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] MFA Rules not processed for user `9cb8e7b3b73947ab92be957f5b4ec5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:44.453565 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:44.522568 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1406/2790] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:44 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:44.534740 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.536839 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:44.598413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca678d-a7c2-4126-b16d-5e983f464745 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.602121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.602738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.603023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.603298 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.612697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.614121 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.623710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1385/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:44 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:44.700385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:44.711071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773512d1-d8a9-41bb-8e79-7019d7cd415a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:44.714734 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b1c76-b755-41e5-8f69-bad57d057fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:44.715345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1c76-b755-41e5-8f69-bad57d057fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:44.715792 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1c76-b755-41e5-8f69-bad57d057fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:44.716077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1c76-b755-41e5-8f69-bad57d057fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:44.721553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:44.782925 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b1c76-b755-41e5-8f69-bad57d057fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:44.784338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b1c76-b755-41e5-8f69-bad57d057fb5 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:44.793220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1407/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:53.955824 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:54.015658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6007-fcad-4318-a812-8d8004699171 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:54.020713 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:54.021340 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:54.021619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:54.021892 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] PATH_INFO: `/v3/users/5380dc2313c142dd952407c5cc74b608` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:54.023039 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5380dc2313c142dd952407c5cc74b608)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:54.024970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:54.070825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5380dc2313c142dd952407c5cc74b608'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.071497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5380dc2313c142dd952407c5cc74b608'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.084812 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5380dc2313c142dd952407c5cc74b608'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.090019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:54.091966 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5380dc2313c142dd952407c5cc74b608'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.092901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5380dc2313c142dd952407c5cc74b608'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.106931 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1386/2793] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:31:53 2022] DELETE /identity/v3/users/5380dc2313c142dd952407c5cc74b608 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:54.111627 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:54.112121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:54.112233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:54.131215 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:54.132464 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:54.193654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1408/2794] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:54.203176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:54.205165 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:54.261296 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6db4e2-9013-4fbe-9245-22c199adae45 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:54.264920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:54.265528 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:54.265815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:54.266087 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:54.275921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:54.277351 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:54.287759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1387/2795] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:54.555814 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:54.557267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:54.571638 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d701c-8e9c-4b18-b220-e71be0b0802f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:54.576647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:54.577298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:54.577569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:54.577843 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] PATH_INFO: `/v3/projects/8185917982a14e1f85285ee8da832473` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:54.580468 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8185917982a14e1f85285ee8da832473)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:54.582779 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:54.630663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8185917982a14e1f85285ee8da832473', 'request_id': 'req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.636332 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8185917982a14e1f85285ee8da832473', 'request_id': 'req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.645036 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8185917982a14e1f85285ee8da832473', 'request_id': 'req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.657586 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] The token cache is being invalidate because project 8185917982a14e1f85285ee8da832473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:31:54.659189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.659189 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.663970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1409/2796] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:31:54 2022] DELETE /identity/v3/projects/8185917982a14e1f85285ee8da832473 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:54.669194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:54.679047 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce56a0-73ac-4ceb-9d14-a9156d0228d3 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:54.740564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:54.741170 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:54.741455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:54.741733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] PATH_INFO: `/v3/users/9cb8e7b3b73947ab92be957f5b4ec5e8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:54.742989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb8e7b3b73947ab92be957f5b4ec5e8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:54.744985 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:54.794017 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb8e7b3b73947ab92be957f5b4ec5e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.794697 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb8e7b3b73947ab92be957f5b4ec5e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.807498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb8e7b3b73947ab92be957f5b4ec5e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.814894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb8e7b3b73947ab92be957f5b4ec5e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.815565 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-576f9bb5-5739-47a4-87e8-f7227b6c7250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb8e7b3b73947ab92be957f5b4ec5e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:54.828455 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1388/2797] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:31:54 2022] DELETE /identity/v3/users/9cb8e7b3b73947ab92be957f5b4ec5e8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:31:55.103345 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.105581 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:55.122274 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d52ad-0f28-4ec6-ae5b-8cc04a47a666 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.154636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:55.185798 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.186450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.186733 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.187013 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] PATH_INFO: `/v3/projects/5e614bd4d8f14a1cb4166f2d23babc13` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.188210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e614bd4d8f14a1cb4166f2d23babc13)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.190153 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.237648 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e614bd4d8f14a1cb4166f2d23babc13', 'request_id': 'req-cf420043-0406-459e-ab6b-0bb419a09653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:55.243058 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e614bd4d8f14a1cb4166f2d23babc13', 'request_id': 'req-cf420043-0406-459e-ab6b-0bb419a09653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:55.250458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e614bd4d8f14a1cb4166f2d23babc13', 'request_id': 'req-cf420043-0406-459e-ab6b-0bb419a09653'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:55.261764 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] The token cache is being invalidate because project 5e614bd4d8f14a1cb4166f2d23babc13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:31:55.262053 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:55.262682 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-cf420043-0406-459e-ab6b-0bb419a09653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:31:55.264220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1410/2798] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:31:55 2022] DELETE /identity/v3/projects/5e614bd4d8f14a1cb4166f2d23babc13 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:55.269200 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.269680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.269781 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.289656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:55.290821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:55.296153 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:55.353712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1389/2799] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:55.362598 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.362887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.363197 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.383148 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:55.384422 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:55.392583 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1411/2800] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:55.403836 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.414493 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0054fbc-80ca-4436-be0e-19a41ac6e0b8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.419244 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.419917 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.420198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.420472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.421194 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:31:55.421545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.423277 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.436247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1390/2801] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:31:55.440897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.450756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc685-83c8-4653-a2d0-dcdc4d3f97bd None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.455985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.456594 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.456878 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.457156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.457818 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.459730 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.487216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1412/2802] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:31:55.491431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.501637 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53648e0b-91d7-418f-8421-7a3a823b17e2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.559333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.560596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.560884 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.561155 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.561867 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.563908 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.585535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1391/2803] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:31:55.589631 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.600432 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb70197-bc0d-438a-b4df-7a14080a6946 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.605006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.605879 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.606157 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.606428 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.607112 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.608741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.618907 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1413/2804] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:55.622894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.633192 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b93e-3b8d-4dd5-b87c-1e0c6c59ec80 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.637797 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1aaeb-9051-49a2-ba8f-d2ee16ce7698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.638398 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1aaeb-9051-49a2-ba8f-d2ee16ce7698 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.638678 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1aaeb-9051-49a2-ba8f-d2ee16ce7698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.638951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1aaeb-9051-49a2-ba8f-d2ee16ce7698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.639682 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1aaeb-9051-49a2-ba8f-d2ee16ce7698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.641294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1aaeb-9051-49a2-ba8f-d2ee16ce7698 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.652901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1392/2805] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:55.657257 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.668260 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5101c6-e116-4153-889b-f7eb9f844e39 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.672838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.673597 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.673883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.674158 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] PATH_INFO: `/v3/projects/e46cc81f3faa404ba1f3528d69acfd50/users/f4b97417cf9c498087984332ac4179e5/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.688572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46cc81f3faa404ba1f3528d69acfd50, user_id=f4b97417cf9c498087984332ac4179e5, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.691201 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.704504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1414/2806] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:55 2022] PUT /identity/v3/projects/e46cc81f3faa404ba1f3528d69acfd50/users/f4b97417cf9c498087984332ac4179e5/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:55.716794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.717095 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.717343 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.747271 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] MFA Rules not processed for user `f4b97417cf9c498087984332ac4179e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:55.748525 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:55.812897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1393/2807] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:55.817761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.875233 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:55.879783 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe5329-2d89-404c-8984-93ebebaecad1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.885647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.886615 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.887133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.887420 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.888333 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.890232 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:55.917305 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1415/2808] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:31:55.921316 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:55.932061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c467b7f-af2d-430e-823a-a0a109ce6016 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:55.935652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:55.991712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:55.992333 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:55.992624 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:55.992899 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:55.993567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:55.995506 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.016986 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1394/2809] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:31:55 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:31:56.021544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.032001 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c4ea5-e49d-4f3b-9279-fa93f73bf3c9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.036639 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.037248 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.037534 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.037813 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.038510 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.040142 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.050626 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1416/2810] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:56.054514 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.064940 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3703-9765-44ca-8e74-60c4e56de0fd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.069793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39d11a-d180-4d30-a8f0-9ea1f6c6deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.070391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d11a-d180-4d30-a8f0-9ea1f6c6deb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.070671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d11a-d180-4d30-a8f0-9ea1f6c6deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.070942 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d11a-d180-4d30-a8f0-9ea1f6c6deb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.071690 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39d11a-d180-4d30-a8f0-9ea1f6c6deb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.073269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39d11a-d180-4d30-a8f0-9ea1f6c6deb9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.083767 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1395/2811] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:56.087767 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.097716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ececf13-aaa7-4285-8818-6757c988b22c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.102350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.103003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.103285 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.103593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] PATH_INFO: `/v3/projects/cf4f36fb43594e929d362a332f8e33cb/users/2d56b66b82864449a1f8e55767c0fe00/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.118307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4f36fb43594e929d362a332f8e33cb, user_id=2d56b66b82864449a1f8e55767c0fe00, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.120993 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.134087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1417/2812] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:56 2022] PUT /identity/v3/projects/cf4f36fb43594e929d362a332f8e33cb/users/2d56b66b82864449a1f8e55767c0fe00/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:56.144827 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.145115 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.145391 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.173350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] MFA Rules not processed for user `2d56b66b82864449a1f8e55767c0fe00`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:56.175143 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:56.238410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1396/2813] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:56.242965 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.306866 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc84fd-c1b7-4b70-ba53-9285aca13bc5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.311561 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.312178 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.312458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.312732 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.313396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.315352 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.341619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1418/2814] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:31:56 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:31:56.346382 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.357258 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c76f2-98ba-49c7-8be3-1c55fea20add None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.412621 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.413239 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.413520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.413796 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.414471 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.416413 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.436944 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1397/2815] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:31:56 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:31:56.440977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.451514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e407-5122-45e2-b56c-b95ebe032ab5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.456629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.457246 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.457531 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.457806 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.458501 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.460135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.471245 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1419/2816] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:56.475410 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.482021 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:56.485888 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a769e-d6c3-4d8b-9657-cfbcfbd46290 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.491657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.492335 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.492617 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.492896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] PATH_INFO: `/v3/projects/7ec5ed126be14bfabd0bce0873df63a6/users/cfee9518ba7f4bea85a438a35b90eaa1/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.507759 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec5ed126be14bfabd0bce0873df63a6, user_id=cfee9518ba7f4bea85a438a35b90eaa1, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.510417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.523897 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1398/2817] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:56 2022] PUT /identity/v3/projects/7ec5ed126be14bfabd0bce0873df63a6/users/cfee9518ba7f4bea85a438a35b90eaa1/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:56.527997 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.538778 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f5e6a-ed91-4507-9a42-38f160ecdef4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.595235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c21cc1-5c99-4277-a14d-ff6c21b0c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.595804 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c21cc1-5c99-4277-a14d-ff6c21b0c130 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.596086 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c21cc1-5c99-4277-a14d-ff6c21b0c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.596357 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c21cc1-5c99-4277-a14d-ff6c21b0c130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.597049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c21cc1-5c99-4277-a14d-ff6c21b0c130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.598633 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c21cc1-5c99-4277-a14d-ff6c21b0c130 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.609339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1420/2818] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:31:56.613498 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.623825 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2457-463b-4609-a2e2-f0e9f8db9136 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.628907 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.629553 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.629833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.630106 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] PATH_INFO: `/v3/projects/7ec5ed126be14bfabd0bce0873df63a6/users/cfee9518ba7f4bea85a438a35b90eaa1/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.632099 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec5ed126be14bfabd0bce0873df63a6, user_id=cfee9518ba7f4bea85a438a35b90eaa1, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.634722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.645586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1399/2819] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:31:56 2022] PUT /identity/v3/projects/7ec5ed126be14bfabd0bce0873df63a6/users/cfee9518ba7f4bea85a438a35b90eaa1/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:31:56.656985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.657358 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.657763 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.685459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] MFA Rules not processed for user `cfee9518ba7f4bea85a438a35b90eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:31:56.686689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:56.692766 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:31:56.756556 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1421/2820] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:31:56.768508 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:56.770430 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:56.827109 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b52eeb-9917-4ee9-91c9-1382e3478b3d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:56.831447 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:56.832094 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:56.832379 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:56.832657 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:56.841955 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:56.843394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:56.852932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1400/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:31:57.366237 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:31:57.367716 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:31:57.379713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f97c36-2923-4a95-8a20-99a0d72ba952 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:31:57.383540 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:31:57.384152 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:31:57.384433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:31:57.384706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:31:57.442789 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:31:57.444251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:31:57.453057 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1422/2822] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:31:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:01.449077 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:01.458515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1695b3-0591-4528-b83f-424dafa6b3fc service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:01.462152 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045684d-2332-4a2e-b620-4d2797f811ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:01.462871 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045684d-2332-4a2e-b620-4d2797f811ee service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:01.463163 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045684d-2332-4a2e-b620-4d2797f811ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:01.463442 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045684d-2332-4a2e-b620-4d2797f811ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:01.520279 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045684d-2332-4a2e-b620-4d2797f811ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:01.521703 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045684d-2332-4a2e-b620-4d2797f811ee service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:01.530519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1401/2823] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:03.588802 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:03.651071 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ecf7-ee05-4244-ae97-e6b4581a6779 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:03.655569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:03.656208 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:03.656488 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:03.656761 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] PATH_INFO: `/v3/users/f4b97417cf9c498087984332ac4179e5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:03.658436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b97417cf9c498087984332ac4179e5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:03.660486 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:03.704361 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:03.707338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b97417cf9c498087984332ac4179e5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:03.708336 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b97417cf9c498087984332ac4179e5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:03.721411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b97417cf9c498087984332ac4179e5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:03.728780 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b97417cf9c498087984332ac4179e5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:03.735903 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b97417cf9c498087984332ac4179e5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:03.742388 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1423/2824] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:03 2022] DELETE /identity/v3/users/f4b97417cf9c498087984332ac4179e5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:03.746981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:03.747478 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:03.747607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:03.766570 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:03.768327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:03.827082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1402/2825] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:03.837135 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:03.838585 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:03.903506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1df2-d110-4844-ad89-0b81bb0ff8cd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:03.907170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:03.907816 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:03.908674 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:03.908943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:03.918062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:03.919492 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:03.929077 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1424/2826] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:04.197707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:04.199435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:04.213658 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782b1ed-3ea2-421b-8f54-137917ad1256 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:04.218545 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:04.219195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:04.219520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:04.219806 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] PATH_INFO: `/v3/projects/e46cc81f3faa404ba1f3528d69acfd50` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:04.221056 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46cc81f3faa404ba1f3528d69acfd50)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:04.222981 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:04.225960 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:04.277330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46cc81f3faa404ba1f3528d69acfd50', 'request_id': 'req-2933059e-4d02-4ed9-869f-efc231191607'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.282846 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46cc81f3faa404ba1f3528d69acfd50', 'request_id': 'req-2933059e-4d02-4ed9-869f-efc231191607'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.290842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46cc81f3faa404ba1f3528d69acfd50', 'request_id': 'req-2933059e-4d02-4ed9-869f-efc231191607'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.302454 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] The token cache is being invalidate because project e46cc81f3faa404ba1f3528d69acfd50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:04.302775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.303403 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.305007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1403/2827] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:04 2022] DELETE /identity/v3/projects/e46cc81f3faa404ba1f3528d69acfd50 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:04.309223 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:04.319166 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c47b19-02a8-4cad-b63e-231b270d3864 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:04.375121 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:04.375708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:04.375996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:04.376273 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] PATH_INFO: `/v3/users/2d56b66b82864449a1f8e55767c0fe00` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:04.377943 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d56b66b82864449a1f8e55767c0fe00)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:04.379888 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:04.425096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d56b66b82864449a1f8e55767c0fe00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.425746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d56b66b82864449a1f8e55767c0fe00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.437683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d56b66b82864449a1f8e55767c0fe00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.439905 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:04.447469 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d56b66b82864449a1f8e55767c0fe00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.457069 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d56b66b82864449a1f8e55767c0fe00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.463172 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1425/2828] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:04 2022] DELETE /identity/v3/users/2d56b66b82864449a1f8e55767c0fe00 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:04.743652 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:04.745443 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:04.762491 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933059e-4d02-4ed9-869f-efc231191607 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:04.821932 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:04.822581 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:04.822862 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:04.823138 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] PATH_INFO: `/v3/projects/cf4f36fb43594e929d362a332f8e33cb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:04.824334 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4f36fb43594e929d362a332f8e33cb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:04.826231 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:04.874801 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4f36fb43594e929d362a332f8e33cb', 'request_id': 'req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.880485 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4f36fb43594e929d362a332f8e33cb', 'request_id': 'req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.888432 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4f36fb43594e929d362a332f8e33cb', 'request_id': 'req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.901314 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] The token cache is being invalidate because project cf4f36fb43594e929d362a332f8e33cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:04.901930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.902891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:04.904620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1404/2829] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:04 2022] DELETE /identity/v3/projects/cf4f36fb43594e929d362a332f8e33cb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:04.909102 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:04.910621 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:04.923330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f4bc4-ace5-4437-ac0e-1ace1425d201 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:04.981417 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:04.982064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:04.982349 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:04.982625 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] PATH_INFO: `/v3/users/cfee9518ba7f4bea85a438a35b90eaa1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:04.983824 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfee9518ba7f4bea85a438a35b90eaa1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:04.985752 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.034271 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfee9518ba7f4bea85a438a35b90eaa1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.034574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfee9518ba7f4bea85a438a35b90eaa1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.046660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfee9518ba7f4bea85a438a35b90eaa1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.055795 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfee9518ba7f4bea85a438a35b90eaa1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.062895 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b47f7cb8-099f-453a-a47f-5ae19aabacb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfee9518ba7f4bea85a438a35b90eaa1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.068887 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1426/2830] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:04 2022] DELETE /identity/v3/users/cfee9518ba7f4bea85a438a35b90eaa1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:05.345035 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.347280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:05.368823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d600-1ba5-4dbe-9854-d2eed2a5d0ce admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.388295 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:05.447567 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.448540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.448965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.449394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] PATH_INFO: `/v3/projects/7ec5ed126be14bfabd0bce0873df63a6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.451280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec5ed126be14bfabd0bce0873df63a6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:05.454518 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.505789 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec5ed126be14bfabd0bce0873df63a6', 'request_id': 'req-7ba528cc-8b8e-47df-b8bb-192face53316'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.511482 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec5ed126be14bfabd0bce0873df63a6', 'request_id': 'req-7ba528cc-8b8e-47df-b8bb-192face53316'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.521969 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec5ed126be14bfabd0bce0873df63a6', 'request_id': 'req-7ba528cc-8b8e-47df-b8bb-192face53316'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.534742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] The token cache is being invalidate because project 7ec5ed126be14bfabd0bce0873df63a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:05.535034 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.535776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7ba528cc-8b8e-47df-b8bb-192face53316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:05.537302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1405/2831] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:05 2022] DELETE /identity/v3/projects/7ec5ed126be14bfabd0bce0873df63a6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:05.543838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.544131 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.544411 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.565549 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:05.566793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:05.635175 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1427/2832] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:32:05.644902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.645406 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.645821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.667707 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:05.668950 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:05.677063 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1406/2833] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:05.681620 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.687075 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:05.694088 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b00340-a840-4e4f-8efe-a90f32e04b87 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.701382 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.702129 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.702458 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.702775 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.703574 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:32:05.703919 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:05.706400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.718459 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1428/2834] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:32:05.722930 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.732947 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56846d-6762-4a7f-8cb0-ec9a455be2e1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.737794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.738473 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.738754 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.739032 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.739742 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:05.741662 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.778027 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1407/2835] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:32:05 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:05.782861 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.799063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e0a4-7153-4fe3-a639-02a66522e934 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.861872 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.862503 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.862788 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.863062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.863787 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:05.865713 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.899139 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1429/2836] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:32:05 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:05.903661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.915417 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f55ab-b80c-4e03-8e6f-b226ebb3b31f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.920110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.920725 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.921007 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.921281 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.921977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:05.923653 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.934195 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1408/2837] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:32:05.938944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.949884 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9038-ba32-42bc-bf63-9c7c8a7fb518 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.954822 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa424a0a-e00c-4d2b-bd9c-0df87aa86c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.956006 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa424a0a-e00c-4d2b-bd9c-0df87aa86c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.956290 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa424a0a-e00c-4d2b-bd9c-0df87aa86c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.956569 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa424a0a-e00c-4d2b-bd9c-0df87aa86c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:05.957267 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa424a0a-e00c-4d2b-bd9c-0df87aa86c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:05.958893 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa424a0a-e00c-4d2b-bd9c-0df87aa86c89 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:05.970709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1430/2838] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:32:05.975275 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:05.986901 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad2ec0-78e1-4b0f-90b5-4b28fbd76149 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:05.991513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:05.992172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:05.992453 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:05.992727 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] PATH_INFO: `/v3/projects/c0dae754020b4b9186c1285de7d0fbbc/users/a8aa0e246fe64f5ea8225700d66d7d94/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.008568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0dae754020b4b9186c1285de7d0fbbc, user_id=a8aa0e246fe64f5ea8225700d66d7d94, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.011386 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.025359 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1409/2839] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:32:05 2022] PUT /identity/v3/projects/c0dae754020b4b9186c1285de7d0fbbc/users/a8aa0e246fe64f5ea8225700d66d7d94/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:06.039338 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.039589 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.039838 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.071286 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] MFA Rules not processed for user `a8aa0e246fe64f5ea8225700d66d7d94`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:06.072547 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:06.138326 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1431/2840] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:06.143134 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.194768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:06.211696 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692ac6-f9d8-453f-8f6c-e5ab7147225c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.216515 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.217174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.217466 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.217753 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.218431 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.220423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.249552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1410/2841] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:32:06 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:06.254553 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.265706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256571e-08fd-48f5-a3b1-f05d64eea966 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.319330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:06.326502 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.327654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.327939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.328216 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.328889 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.330839 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.354675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1432/2842] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:32:06 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:06.359597 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.371276 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fb854-e55b-44ef-8059-2e4568ba97af admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.376051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.376680 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.376959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.377235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.377937 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.379587 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.391350 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1411/2843] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:32:06.395499 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.405572 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0c7c-8374-4c3f-8472-84ec09300e35 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.410758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729705-453a-4a2f-bcf9-1352388081be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.411366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729705-453a-4a2f-bcf9-1352388081be admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.411683 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729705-453a-4a2f-bcf9-1352388081be admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.411961 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729705-453a-4a2f-bcf9-1352388081be admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.412655 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e729705-453a-4a2f-bcf9-1352388081be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.414253 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e729705-453a-4a2f-bcf9-1352388081be admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.424970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1433/2844] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:32:06.429189 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.439645 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acf5cb-2689-41ea-b603-3e5f97f14003 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.444684 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.445360 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.445643 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.445920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] PATH_INFO: `/v3/projects/bdc7768222f146859c7b8a913ee32cc8/users/69cdae733bd94919ac8a4468044b8e00/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.461257 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc7768222f146859c7b8a913ee32cc8, user_id=69cdae733bd94919ac8a4468044b8e00, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.463974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.478876 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1412/2845] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:32:06 2022] PUT /identity/v3/projects/bdc7768222f146859c7b8a913ee32cc8/users/69cdae733bd94919ac8a4468044b8e00/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:06.490133 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.490426 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.490708 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.518660 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] MFA Rules not processed for user `69cdae733bd94919ac8a4468044b8e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:06.519970 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:06.586425 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1434/2846] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:06.591049 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.672607 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ed392-68e5-4ab7-865d-2a2043a461f2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.678305 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.678948 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.679234 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.679540 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.680221 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.682404 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.711661 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1413/2847] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:32:06 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:06.715771 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.727307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab5b32-d73a-4496-9585-9013371e2d90 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.793230 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.793869 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.794155 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.794433 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.795127 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.797124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.819064 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1435/2848] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:32:06 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:06.831660 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.843082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f4cfa-fb23-4aa9-845d-31dfba49e240 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.851791 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.853078 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.853551 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.854164 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.855280 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.858552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.876894 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1414/2849] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:32:06.881125 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.891786 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74babd22-8cd6-46ad-9fb5-0565d69d0cef admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:06.898294 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:06.899163 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:06.899391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:06.899704 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] PATH_INFO: `/v3/projects/9d831d604a8141369e8d74c7697d1eaf/users/c184b55e735742ffb4607489e6693073/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:06.917902 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d831d604a8141369e8d74c7697d1eaf, user_id=c184b55e735742ffb4607489e6693073, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:06.920494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:06.938378 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1436/2850] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:32:06 2022] PUT /identity/v3/projects/9d831d604a8141369e8d74c7697d1eaf/users/c184b55e735742ffb4607489e6693073/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:06.940919 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:06.959282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360590-6b08-4965-b235-78c6ee38e4cf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:07.027736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e3394-4d1c-4138-8043-3efd9079e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:07.028802 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e3394-4d1c-4138-8043-3efd9079e36e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:07.029293 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e3394-4d1c-4138-8043-3efd9079e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:07.029794 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e3394-4d1c-4138-8043-3efd9079e36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:07.031080 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e3394-4d1c-4138-8043-3efd9079e36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:07.034821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e3394-4d1c-4138-8043-3efd9079e36e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:07.054777 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1415/2851] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:32:07.059544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:07.070675 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1efd5-31de-4ba0-bd0e-7a020c5ab713 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:07.075747 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:07.076424 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:07.076706 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:07.077031 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] PATH_INFO: `/v3/projects/9d831d604a8141369e8d74c7697d1eaf/users/c184b55e735742ffb4607489e6693073/roles/92897736a4464cec9c4a1b3a24b138b1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:07.078977 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d831d604a8141369e8d74c7697d1eaf, user_id=c184b55e735742ffb4607489e6693073, role_id=92897736a4464cec9c4a1b3a24b138b1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:07.082514 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:07.094067 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1437/2852] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:32:07 2022] PUT /identity/v3/projects/9d831d604a8141369e8d74c7697d1eaf/users/c184b55e735742ffb4607489e6693073/roles/92897736a4464cec9c4a1b3a24b138b1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:32:07.104445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:07.104739 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:07.105019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:07.130920 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:07.148548 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] MFA Rules not processed for user `c184b55e735742ffb4607489e6693073`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:07.149778 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:07.220868 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1416/2853] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:07.239022 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:07.241409 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:07.269050 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:07.317061 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097569c-5cde-40d1-834b-5b2e7c453096 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:07.321515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:07.322593 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:07.322840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:07.323209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:07.333849 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:07.335317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:07.345304 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1438/2854] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:07.387248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:07.388718 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:07.402120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de42740-a593-4fd5-bad6-3787373a72c6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:07.405823 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3cb88-9942-4fb9-8c55-e4da99bb63eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:07.406444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3cb88-9942-4fb9-8c55-e4da99bb63eb service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:07.406724 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3cb88-9942-4fb9-8c55-e4da99bb63eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:07.406996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3cb88-9942-4fb9-8c55-e4da99bb63eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:07.465750 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3cb88-9942-4fb9-8c55-e4da99bb63eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:07.467197 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3cb88-9942-4fb9-8c55-e4da99bb63eb service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:07.487260 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1417/2855] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:10.413399 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:10.478591 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d936fbf-7d64-4cee-be71-3b31931a0d41 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:10.483515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:10.484170 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:10.484448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:10.484726 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] PATH_INFO: `/v3/users/a8aa0e246fe64f5ea8225700d66d7d94` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:10.485945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8aa0e246fe64f5ea8225700d66d7d94)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:10.488109 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:10.539504 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8aa0e246fe64f5ea8225700d66d7d94'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:10.540192 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8aa0e246fe64f5ea8225700d66d7d94'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:10.552741 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8aa0e246fe64f5ea8225700d66d7d94'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:10.560439 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8aa0e246fe64f5ea8225700d66d7d94'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:10.567602 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8aa0e246fe64f5ea8225700d66d7d94'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:10.573563 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1439/2856] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:10 2022] DELETE /identity/v3/users/a8aa0e246fe64f5ea8225700d66d7d94 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:10.578119 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:10.578634 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:10.578746 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:10.598997 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:10.600248 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:10.668589 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1418/2857] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:32:10.679003 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:10.680632 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:10.738909 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:10.758644 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a93fe-a91c-4b35-a423-5dd806672810 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:10.762669 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:10.763302 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:10.763618 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:10.763918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:10.774124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:10.775601 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:10.786691 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1440/2858] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:11.086935 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:11.088415 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:11.105573 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cea73-d2dc-4812-82dc-14adeb6f0b9e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:11.114446 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:11.115497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:11.115970 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:11.116448 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] PATH_INFO: `/v3/projects/c0dae754020b4b9186c1285de7d0fbbc` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:11.118247 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0dae754020b4b9186c1285de7d0fbbc)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:11.121648 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:11.160131 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:11.177267 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0dae754020b4b9186c1285de7d0fbbc', 'request_id': 'req-ca27448c-9a98-4799-974c-e8f9122d3717'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.183223 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0dae754020b4b9186c1285de7d0fbbc', 'request_id': 'req-ca27448c-9a98-4799-974c-e8f9122d3717'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.192520 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0dae754020b4b9186c1285de7d0fbbc', 'request_id': 'req-ca27448c-9a98-4799-974c-e8f9122d3717'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.204537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] The token cache is being invalidate because project c0dae754020b4b9186c1285de7d0fbbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:11.205388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.206046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.207576 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1419/2859] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:11 2022] DELETE /identity/v3/projects/c0dae754020b4b9186c1285de7d0fbbc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:11.211512 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:11.221958 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fc431-f719-472b-96d4-c36135624876 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:11.284401 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:11.285049 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:11.285337 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:11.285619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] PATH_INFO: `/v3/users/69cdae733bd94919ac8a4468044b8e00` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:11.286801 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=69cdae733bd94919ac8a4468044b8e00)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:11.288773 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:11.345085 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69cdae733bd94919ac8a4468044b8e00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.346507 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69cdae733bd94919ac8a4468044b8e00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.360386 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69cdae733bd94919ac8a4468044b8e00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.370671 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69cdae733bd94919ac8a4468044b8e00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.378235 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69cdae733bd94919ac8a4468044b8e00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.384474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1441/2860] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:11 2022] DELETE /identity/v3/users/69cdae733bd94919ac8a4468044b8e00 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:11.718294 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:11.720965 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:11.740738 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27448c-9a98-4799-974c-e8f9122d3717 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:11.813424 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:11.814079 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:11.814399 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:11.814675 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] PATH_INFO: `/v3/projects/bdc7768222f146859c7b8a913ee32cc8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:11.815895 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc7768222f146859c7b8a913ee32cc8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:11.817814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:11.866996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc7768222f146859c7b8a913ee32cc8', 'request_id': 'req-178c31c4-f14d-4644-b0f6-0f82e0080623'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.872620 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc7768222f146859c7b8a913ee32cc8', 'request_id': 'req-178c31c4-f14d-4644-b0f6-0f82e0080623'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.880929 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc7768222f146859c7b8a913ee32cc8', 'request_id': 'req-178c31c4-f14d-4644-b0f6-0f82e0080623'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.892730 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] The token cache is being invalidate because project bdc7768222f146859c7b8a913ee32cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:11.893024 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.893686 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:11.895214 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1420/2861] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:11 2022] DELETE /identity/v3/projects/bdc7768222f146859c7b8a913ee32cc8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:11.899234 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:11.900768 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:11.914670 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53840b2-5303-4739-a095-74cb911823bb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:11.979807 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:11.980410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:11.980696 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:11.981242 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] PATH_INFO: `/v3/users/c184b55e735742ffb4607489e6693073` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:11.982474 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c184b55e735742ffb4607489e6693073)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:11.984515 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:11.987339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:12.037259 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c184b55e735742ffb4607489e6693073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.037918 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c184b55e735742ffb4607489e6693073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.050035 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c184b55e735742ffb4607489e6693073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.059251 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c184b55e735742ffb4607489e6693073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.066944 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c200cb55-7356-47e5-afc5-07165b755c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c184b55e735742ffb4607489e6693073'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.073147 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1442/2862] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:11 2022] DELETE /identity/v3/users/c184b55e735742ffb4607489e6693073 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:12.388646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.390896 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:12.407592 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c31c4-f14d-4644-b0f6-0f82e0080623 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:12.459557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:12.476575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:12.477230 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.477519 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.477833 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] PATH_INFO: `/v3/projects/9d831d604a8141369e8d74c7697d1eaf` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.479044 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d831d604a8141369e8d74c7697d1eaf)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:12.481377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:12.537844 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d831d604a8141369e8d74c7697d1eaf', 'request_id': 'req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.544038 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d831d604a8141369e8d74c7697d1eaf', 'request_id': 'req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.552174 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d831d604a8141369e8d74c7697d1eaf', 'request_id': 'req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.565019 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] The token cache is being invalidate because project 9d831d604a8141369e8d74c7697d1eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:12.565310 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.565954 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3c3b1bea-807a-4c8d-b8b1-64a6e13dbe70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:12.567488 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1421/2863] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:12 2022] DELETE /identity/v3/projects/9d831d604a8141369e8d74c7697d1eaf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:12.575963 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.576255 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.576538 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.597156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:12.598397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:12.665703 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1443/2864] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:12.676982 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.677272 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.677552 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.699513 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:12.700722 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:12.709878 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1422/2865] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:12.714317 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.724528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432864c-f587-4fd8-bbce-b19213bd23fa None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:12.729366 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:12.729999 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.730275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.730546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.731684 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:32:12.731933 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:12.733559 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:12.746475 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1444/2866] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Aug 1 14:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:32:12.751646 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.762775 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a81d-2831-499d-a1d1-b8c2eb8dc941 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:12.767717 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:12.768336 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.768619 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.768891 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.769557 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:12.771450 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:12.801855 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1423/2867] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Aug 1 14:32:12 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:12.806207 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.816986 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1fa5d-8870-497d-862d-934f1bd93a35 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:12.878493 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:12.879126 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.879414 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.879724 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.880410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:12.882350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:12.899753 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:12.907017 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1445/2868] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Aug 1 14:32:12 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:32:12.912977 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.924438 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0391c-42c2-45ed-a1d1-0f949aa6878b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:12.929157 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:12.929776 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.930061 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.930338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.931052 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:12.932729 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:12.944853 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1424/2869] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:32:12.949089 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.959397 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f506816-c829-4c76-bb22-5e915188114b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:12.964697 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cbecd-da30-4cdc-81e1-d7f53ef5a5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:12.965330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cbecd-da30-4cdc-81e1-d7f53ef5a5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:12.965616 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cbecd-da30-4cdc-81e1-d7f53ef5a5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:12.965897 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cbecd-da30-4cdc-81e1-d7f53ef5a5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:12.966600 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cbecd-da30-4cdc-81e1-d7f53ef5a5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:12.968258 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cbecd-da30-4cdc-81e1-d7f53ef5a5b0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:12.980099 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1446/2870] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Aug 1 14:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:32:12.985147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:12.996474 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f9144-372c-45ea-9483-1444c06e13ed admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:13.001445 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:13.002721 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:13.003015 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:13.003292 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] PATH_INFO: `/v3/projects/961778216ec84b93b3ba76fbad2af11f/users/083067de56b4452f9ba26fd79bca65b0/roles/d47409cb4aa642018317ecedd3a36461` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:13.019556 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961778216ec84b93b3ba76fbad2af11f, user_id=083067de56b4452f9ba26fd79bca65b0, role_id=d47409cb4aa642018317ecedd3a36461)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:13.022265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:13.036733 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1425/2871] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Aug 1 14:32:12 2022] PUT /identity/v3/projects/961778216ec84b93b3ba76fbad2af11f/users/083067de56b4452f9ba26fd79bca65b0/roles/d47409cb4aa642018317ecedd3a36461 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:32:13.047758 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:13.048045 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:13.048330 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:13.079762 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] MFA Rules not processed for user `083067de56b4452f9ba26fd79bca65b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:13.080985 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:13.151066 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1447/2872] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:13.163883 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:13.165644 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:13.225388 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab95c1-3a97-4d0e-9502-f91b689c8303 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:13.229120 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad26f19-bcd3-42be-a3c4-86d01ac156dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a7a9a0bda04d128a4f045fdb0bc80c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:13.229748 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad26f19-bcd3-42be-a3c4-86d01ac156dd service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:13.230030 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad26f19-bcd3-42be-a3c4-86d01ac156dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:13.230302 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad26f19-bcd3-42be-a3c4-86d01ac156dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:13.240176 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad26f19-bcd3-42be-a3c4-86d01ac156dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:13.241535 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad26f19-bcd3-42be-a3c4-86d01ac156dd service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:13.251683 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1426/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:15.842220 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:15.907410 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68781d52-0356-4cae-968a-ceaad24a229f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:15.912090 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:15.912681 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:15.912962 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:15.913240 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] PATH_INFO: `/v3/users/083067de56b4452f9ba26fd79bca65b0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:15.914423 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083067de56b4452f9ba26fd79bca65b0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:15.916394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:15.966605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083067de56b4452f9ba26fd79bca65b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:15.967262 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083067de56b4452f9ba26fd79bca65b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:15.983546 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083067de56b4452f9ba26fd79bca65b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:15.993687 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083067de56b4452f9ba26fd79bca65b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:15.999537 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:16.002396 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083067de56b4452f9ba26fd79bca65b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:16.010123 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1448/2874] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:32:15 2022] DELETE /identity/v3/users/083067de56b4452f9ba26fd79bca65b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:16.014939 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:16.015501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:16.015501 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:16.035037 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] MFA Rules not processed for user `65c8d33b0a3b4ecb82a05c93cec78423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:16.036284 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:16.046057 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:16.099714 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1427/2875] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:16.112105 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:16.113558 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:16.183450 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43cf90-d7cf-49f6-8c42-fc4273d38362 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:16.188400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0968445-11be-42fb-9f1f-6e02b815a302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:16.189070 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968445-11be-42fb-9f1f-6e02b815a302 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:16.189465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968445-11be-42fb-9f1f-6e02b815a302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:16.189896 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968445-11be-42fb-9f1f-6e02b815a302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:16.200091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0968445-11be-42fb-9f1f-6e02b815a302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:16.201619 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0968445-11be-42fb-9f1f-6e02b815a302 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:16.211663 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1449/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:32:16.500559 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:16.501996 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:16.515902 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ef74a-c265-4ded-a137-dfb5858cc45a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:16.521101 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:16.521877 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:16.522710 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:16.523005 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] PATH_INFO: `/v3/projects/961778216ec84b93b3ba76fbad2af11f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:16.524756 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=961778216ec84b93b3ba76fbad2af11f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:16.526693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:16.585869 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961778216ec84b93b3ba76fbad2af11f', 'request_id': 'req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:16.591436 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961778216ec84b93b3ba76fbad2af11f', 'request_id': 'req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:16.600086 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961778216ec84b93b3ba76fbad2af11f', 'request_id': 'req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:16.612842 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] The token cache is being invalidate because project 961778216ec84b93b3ba76fbad2af11f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:32:16.613133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:16.613784 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:32:16.615324 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1428/2877] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:32:16 2022] DELETE /identity/v3/projects/961778216ec84b93b3ba76fbad2af11f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:32:26.207137 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=28 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:26.609636 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:30.400087 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:30.400587 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:30.400718 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:30.420214 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] MFA Rules not processed for user `540bec8413b144889a07e9d9458b298e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:32:30.421449 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:30.482118 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1450/2878] 10.222.0.17 () {58 vars in 958 bytes} [Mon Aug 1 14:32:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:32:40.475191 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=25 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:32:58.743427 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:32:58.745745 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:58.838516 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2b8a2-7ac5-4c80-b57b-96bf5b1365f0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:32:58.844483 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:32:58.845113 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:32:58.845394 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:32:58.845671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:32:58.847564 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:32:58.866712 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:32:58.868081 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:32:58.882472 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1429/2879] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:32:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:33:01.984062 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:33:01.985811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:33:02.063350 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d470c-830c-4b02-a702-e06f456daa0d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:33:02.068032 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:33:02.068629 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:33:02.068910 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:33:02.069209 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] PATH_INFO: `/v3/users/6040773eefe146a0adbc7858c7371cb8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:33:02.070465 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6040773eefe146a0adbc7858c7371cb8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:33:02.072436 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:33:02.123901 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6040773eefe146a0adbc7858c7371cb8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.124511 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6040773eefe146a0adbc7858c7371cb8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.137969 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6040773eefe146a0adbc7858c7371cb8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.146063 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6040773eefe146a0adbc7858c7371cb8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.154124 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6040773eefe146a0adbc7858c7371cb8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.162339 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1451/2880] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:33:01 2022] DELETE /identity/v3/users/6040773eefe146a0adbc7858c7371cb8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:33:02.347402 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:33:02.364843 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993c8cf-9d6c-476b-abb3-34aee9ec9919 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:33:02.427033 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:33:02.427706 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:33:02.427989 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:33:02.428263 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] PATH_INFO: `/v3/projects/2a6eb59bc9a445cc9f5b4d3059fbb7f1` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:33:02.430023 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6eb59bc9a445cc9f5b4d3059fbb7f1)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:33:02.431959 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:33:02.486270 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6eb59bc9a445cc9f5b4d3059fbb7f1', 'request_id': 'req-f13992ac-4cb3-41a9-8ec0-092b0f726171'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.492507 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6eb59bc9a445cc9f5b4d3059fbb7f1', 'request_id': 'req-f13992ac-4cb3-41a9-8ec0-092b0f726171'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.501269 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6eb59bc9a445cc9f5b4d3059fbb7f1', 'request_id': 'req-f13992ac-4cb3-41a9-8ec0-092b0f726171'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.513237 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] The token cache is being invalidate because project 2a6eb59bc9a445cc9f5b4d3059fbb7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:33:02.513537 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.514235 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.515815 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1430/2881] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:33:02 2022] DELETE /identity/v3/projects/2a6eb59bc9a445cc9f5b4d3059fbb7f1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:33:02.519898 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:33:02.521359 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:33:02.535021 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fc557-5d0e-4df8-94ac-7b0331c00238 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:33:02.595005 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:33:02.595636 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:33:02.595924 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:33:02.596210 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] PATH_INFO: `/v3/users/5f361d19b28c49e8b406bd0760a08633` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:33:02.597434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f361d19b28c49e8b406bd0760a08633)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:33:02.599391 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:33:02.623434 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:33:02.651266 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f361d19b28c49e8b406bd0760a08633'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.651987 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f361d19b28c49e8b406bd0760a08633'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.666161 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f361d19b28c49e8b406bd0760a08633'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.674256 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f361d19b28c49e8b406bd0760a08633'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.683120 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f361d19b28c49e8b406bd0760a08633'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:02.689264 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1452/2882] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:33:02 2022] DELETE /identity/v3/users/5f361d19b28c49e8b406bd0760a08633 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:33:02.869772 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:33:02.871376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:33:02.887909 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13992ac-4cb3-41a9-8ec0-092b0f726171 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:33:02.910671 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:33:02.957330 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:33:02.957984 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:33:02.958265 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:33:02.958568 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] PATH_INFO: `/v3/projects/822f27f0d71f46809595eba71e693250` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:33:02.960338 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=822f27f0d71f46809595eba71e693250)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:33:02.962327 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:33:03.015593 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822f27f0d71f46809595eba71e693250', 'request_id': 'req-31e9f591-1660-412d-be54-02afdb043dcd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:03.021941 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822f27f0d71f46809595eba71e693250', 'request_id': 'req-31e9f591-1660-412d-be54-02afdb043dcd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:03.030685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822f27f0d71f46809595eba71e693250', 'request_id': 'req-31e9f591-1660-412d-be54-02afdb043dcd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:03.043822 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] The token cache is being invalidate because project 822f27f0d71f46809595eba71e693250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:33:03.044110 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:03.044768 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:33:03.046282 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1431/2883] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:33:02 2022] DELETE /identity/v3/projects/822f27f0d71f46809595eba71e693250 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:33:12.685756 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:33:13.040596 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:34:16.791709 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:16.793823 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:16.875605 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b29ea1-cab4-432d-b1f1-537799da6991 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:16.935316 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:16.935883 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:16.936168 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:16.936448 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:16.938275 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:16.952624 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:16.954592 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:16.966293 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1453/2884] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:34:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:34:20.031687 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:20.033182 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:20.093210 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9f591-1660-412d-be54-02afdb043dcd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:20.097133 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:20.097763 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:20.098051 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:20.098329 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:20.100154 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:20.159736 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:20.161190 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:20.170198 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1432/2885] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:34:26.961394 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=58 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:34:30.166092 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=48 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:34:34.854945 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:34.857156 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:34.869385 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f001a5-3cd2-432e-8a94-34ed29b3f78c service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:34.873110 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771f49a069014fdc956e073c2d0a0fdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:34.873746 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:34.874025 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:34.874298 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:34.876117 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:34.950793 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:34.952307 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:34.962923 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1454/2886] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:34:38.125693 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:38.127376 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:38.144974 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1d4cf-e166-4e1f-9486-b5426204e357 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:38.151066 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:38.151798 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:38.152082 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:38.152423 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:38.154400 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:38.169814 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:38.173575 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:38.187623 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1433/2887] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:34:41.177494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:41.179951 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:41.196868 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b24be-c102-4fff-84d5-14e1a5d6a2b5 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:41.203554 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:41.204441 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:41.204984 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:41.205494 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] PATH_INFO: `/v3/users/540bec8413b144889a07e9d9458b298e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:41.207096 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] RBAC: Authorizing `identity:delete_user(user_id=540bec8413b144889a07e9d9458b298e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:41.209528 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:41.266976 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '540bec8413b144889a07e9d9458b298e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.267820 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '540bec8413b144889a07e9d9458b298e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.287400 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '540bec8413b144889a07e9d9458b298e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.300218 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '540bec8413b144889a07e9d9458b298e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.307959 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '540bec8413b144889a07e9d9458b298e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.315506 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1455/2888] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:34:41 2022] DELETE /identity/v3/users/540bec8413b144889a07e9d9458b298e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:34:41.506313 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:41.517638 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf1a3-11fd-47cd-a42b-5131d71d77f9 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:41.582172 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:41.582821 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:41.583104 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:41.583377 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] PATH_INFO: `/v3/projects/47e500c5985743a2a5abfb99c0f42ab7` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:41.584586 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e500c5985743a2a5abfb99c0f42ab7)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:41.587046 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:41.635854 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e500c5985743a2a5abfb99c0f42ab7', 'request_id': 'req-9fe55677-206e-47d7-bbd7-ced4a77f8de9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.641461 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e500c5985743a2a5abfb99c0f42ab7', 'request_id': 'req-9fe55677-206e-47d7-bbd7-ced4a77f8de9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.650321 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e500c5985743a2a5abfb99c0f42ab7', 'request_id': 'req-9fe55677-206e-47d7-bbd7-ced4a77f8de9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.662656 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] The token cache is being invalidate because project 47e500c5985743a2a5abfb99c0f42ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:34:41.662953 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.663698 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.665858 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1434/2889] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:34:41 2022] DELETE /identity/v3/projects/47e500c5985743a2a5abfb99c0f42ab7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:34:41.670342 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:41.672362 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:41.688751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827660c0-6bfb-46c1-8abd-abb9e838501f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:41.745526 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:41.746128 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:41.746413 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:41.746689 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] PATH_INFO: `/v3/users/c0d20c41cd784174b7890a788a094f37` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:41.747894 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d20c41cd784174b7890a788a094f37)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:41.749811 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:41.755347 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:34:41.797996 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d20c41cd784174b7890a788a094f37'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.798654 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d20c41cd784174b7890a788a094f37'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.811091 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d20c41cd784174b7890a788a094f37'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.818939 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d20c41cd784174b7890a788a094f37'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.826543 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d20c41cd784174b7890a788a094f37'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:41.833647 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1456/2890] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Aug 1 14:34:41 2022] DELETE /identity/v3/users/c0d20c41cd784174b7890a788a094f37 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:34:42.006921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:34:42.008497 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:34:42.021688 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55677-206e-47d7-bbd7-ced4a77f8de9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:34:42.081495 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c8d33b0a3b4ecb82a05c93cec78423', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '045f384688624ccf8339b3c71e2bd890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:34:42.082147 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:34:42.082444 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:34:42.082720 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] PATH_INFO: `/v3/projects/b586502dc95e4aa4af33b77d3466cb92` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:34:42.083951 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b586502dc95e4aa4af33b77d3466cb92)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:34:42.085943 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:34:42.107921 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:34:42.140504 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b586502dc95e4aa4af33b77d3466cb92', 'request_id': 'req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:42.146529 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b586502dc95e4aa4af33b77d3466cb92', 'request_id': 'req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:42.154241 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b586502dc95e4aa4af33b77d3466cb92', 'request_id': 'req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:42.166502 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] The token cache is being invalidate because project b586502dc95e4aa4af33b77d3466cb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:34:42.166793 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:42.167435 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7f75da5a-f067-4c7b-ada4-45b0072b1e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:34:42.168976 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1435/2891] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Aug 1 14:34:42 2022] DELETE /identity/v3/projects/b586502dc95e4aa4af33b77d3466cb92 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:34:51.828992 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:34:52.162685 dv-cinder-845990-5-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:39:21.442809 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:39:21.444272 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:39:21.580182 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8d2a-74e4-4c9a-8977-fca3a5857833 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:39:21.643544 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '324fc1c2ea18420eb01d8b65683818ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f106ff2ec5cb4ecda3cd56c2391bc200', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:39:21.644186 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:39:21.644471 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:39:21.644751 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:39:21.646532 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:39:21.659991 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:39:21.661840 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012590f-51a2-46a9-81fe-ce1d29f0bdb0 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:39:21.673278 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1457/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Aug 1 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:39:31.668845 dv-cinder-845990-5-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}